pipe函数

pipe函数用于创建管道。

The pipe Function

The pipe function is used to create a pipe.

1
2
#include <unistd.h>
int pipe(int fd[2]);

pipe函数的参数是一个包含两个int型整数的数组指针。该函数成功时返回0，并将一对打开的文件描述符值填入其参数指向的数组。如果失败，则返回-1并设置errno。

通过pipe函数创建的这两个文件描述符fd[0]和fd[1]分别构成管道的两端，往fd[1]写入的数据可以从fd[0]读出。并且，fd[0]只能用于从管道读出数据，fd[1]则只能用于往管道写入数据，而不能反过来使用。如果要实现双向的数据传输，就应该使用两个管道。默认情况下，这一对文件描述符都是阻塞的。此时如果我们用read系统调用来读取一个空的管道，则read将被阻塞，直到管道内有数据可读；如果我们用write系统调用来往一个满的管道中写入数据，则write亦将被阻塞，直到管道有足够多的空闲空间可用。但如果应用程序将fd[0]和fd[1]都设置为非阻塞的，则read和write会有不同的行为。

此外，socket的基础API中有一个socketpair函数。它能够方便地创建双向管道。其定义如下：

The pipe function takes a pointer to an array of two ints. On success it returns 0 and fills the array pointed to by its argument with a pair of open file descriptors. On failure it returns -1 and sets errno.

The two file descriptors fd[0] and fd[1] created by the pipe function form the two ends of the pipe: data written to fd[1] can be read from fd[0]. Moreover, fd[0] can only be used to read data from the pipe, and fd[1] can only be used to write data to the pipe; it cannot be used the other way around. To achieve bidirectional data transfer, two pipes should be used. By default, this pair of file descriptors is blocking. In this case, if we use the read system call to read from an empty pipe, read will block until there is data available in the pipe; if we use the write system call to write to a full pipe, write will also block until the pipe has enough free space. However, if the application sets both fd[0] and fd[1] to non-blocking, read and write will behave differently.

In addition, there is a socketpair function in the basic socket API. It can conveniently create a bidirectional pipe. Its definition is as follows:

1
2
3
#include <sys/types.h>
#include <sys/socket.h>
int socketpair(int domain,int type,int protocol,int fd[2]);

socketpair前三个参数的含义与socket系统调用的三个参数完全相同，但domain只能使用UNIX本地域协议族AF_UNIX，因为我们仅能在本地使用这个双向管道。最后一个参数则和pipe系统调用的参数一样，只不过socketpair创建的这对文件描述符都是既可读又可写的。socketpair成功时返回0，失败时返回-1并设置errno。

dup函数和dup2函数

有时我们希望把标准输入重定向到一个文件，或者把标准输出重定向到一个网络连接（比如CGI编程）。这可以通过下面的用于复制文件描述符的dup或dup2函数来实现：

The dup and dup2 Functions

Sometimes we want to redirect standard input to a file, or redirect standard output to a network connection (e.g., in CGI programming). This can be done with the dup or dup2 functions below, which are used to duplicate file descriptors:

1
2
3
#include <unistd.h>
int dup(int file_descriptor);
int dup2(int file_descriptor_one,int file_descriptor_two);

dup函数创建一个新的文件描述符，该新文件描述符和原有文件描述符file_descriptor指向相同的文件、管道或者网络连接。并且dup返回的文件描述符总是取系统当前可用的最小整数值。dup2和dup类似，不过它将返回第一个不小于file_descriptor_two的整数值。dup和dup2系统调用失败时返回-1并设置errno。

The dup function creates a new file descriptor that points to the same file, pipe, or network connection as the original file descriptor file_descriptor. Moreover, the file descriptor returned by dup is always the smallest integer value currently available in the system. dup2 is similar to dup, but it returns the first integer value not less than file_descriptor_two. On failure, the dup and dup2 system calls return -1 and set errno.

注意　通过dup和dup2创建的文件描述符并不继承原文件描述符的属性，比如close-on-exec和non-blocking等。

Note that the file descriptors created by dup and dup2 do not inherit the attributes of the original file descriptor, such as close-on-exec and non-blocking.

readv函数和writev函数

readv函数将数据从文件描述符读到分散的内存块中，即分散读；writev函数则将多块分散的内存数据一并写入文件描述符中，即集中写。它们的定义如下：

The readv and writev Functions

The readv function reads data from a file descriptor into scattered memory blocks, i.e., scatter read; the writev function writes multiple scattered memory blocks of data into a file descriptor at once, i.e., gather write. Their definitions are as follows:

1
2
3
#include <sys/uio.h>
ssize_t readv(int fd,const struct iovec* vector,int count);
ssize_t writev(int fd,const struct iovec* vector,int count);

fd参数是被操作的目标文件描述符。vector参数的类型是iovec结构数组。该结构体描述一块内存区，count参数是vector数组的长度，即有多少块内存数据需要从fd读出或写到fd。readv和writev在成功时返回读出/写入fd的字节数，失败则返回-1并设置errno。

从 vector 结构体数组中 读取/写入 数据到文件描述符 fd，count 是结构体数组的大小。结构体数组中的每个结构体包括 iov_base 成员和 iov_len 成员，iov_base 成员指向内存地址，iov_len 表示这块地址的长度。

The fd parameter is the target file descriptor to operate on. The vector parameter is an array of iovec structures. This structure describes a memory area, and the count parameter is the length of the vector array, i.e., how many blocks of memory data need to be read from or written to fd. On success, readv and writev return the number of bytes read from / written to fd; on failure they return -1 and set errno.

Read/write data from/to the file descriptor fd from the vector structure array; count is the size of the structure array. Each structure in the array includes the iov_base member and the iov_len member; the iov_base member points to a memory address, and iov_len indicates the length of this address.

1
2
3
4
5
6
#include <sys/uio.h>

struct iovec{
    ptr_t iov_base; // 起始地址
    size_t iov_len; // 字节数
}

sendfile 函数

sendfile函数在两个文件描述符之间直接传递数据（完全在内核中操作），从而避免了内核缓冲区和用户缓冲区之间的数据拷贝，效率很高，这被称为零拷贝。sendfile函数的定义如下：

The sendfile Function

The sendfile function transfers data directly between two file descriptors (operating entirely in the kernel), thus avoiding the data copy between the kernel buffer and the user buffer. It is very efficient, and this is called zero-copy. The definition of the sendfile function is as follows:

1
2
#include <sys/sendfile.h>
ssize_t sendfile(int out_fd,int in_fd,off_t* offset,size_t count);

in_fd参数是待读出内容的文件描述符，out_fd参数是待写入内容的文件描述符。offset参数指定从读入文件流的哪个位置开始读，如果为空，则使用读入文件流默认的起始位置。count参数指定在文件描述符in_fd和out_fd之间传输的字节数。sendfile成功时返回传输的字节数，失败则返回-1并设置errno。该函数的man手册明确指出，in_fd必须是一个支持类似mmap函数的文件描述符，即它必须指向真实的文件，不能是socket和管道；而out_fd则必须是一个socket。由此可见，sendfile几乎是专门为在网络上传输文件而设计的。

从 in_fd 读取数据写入到 out_fd，第三个参数是读取的偏移量，第四个参数是字节数。

The in_fd parameter is the file descriptor to read content from, and the out_fd parameter is the file descriptor to write content to. The offset parameter specifies the position in the input file stream from which to start reading; if it is NULL, the default start position of the input file stream is used. The count parameter specifies the number of bytes to transfer between file descriptors in_fd and out_fd. On success, sendfile returns the number of bytes transferred; on failure it returns -1 and sets errno. The man page of this function explicitly states that in_fd must be a file descriptor that supports functions like mmap, i.e., it must point to a real file and cannot be a socket or a pipe; while out_fd must be a socket. It can be seen that sendfile is almost specifically designed for transferring files over the network.

Read data from in_fd and write it to out_fd; the third parameter is the read offset, and the fourth parameter is the number of bytes.

mmap函数和munmap函数

mmap函数用于申请一段内存空间。我们可以将这段内存作为进程间通信的共享内存，也可以将文件直接映射到其中。munmap函数则释放由mmap创建的这段内存空间。它们的定义如下：

The mmap and munmap Functions

The mmap function is used to allocate a segment of memory. We can use this memory as shared memory for inter-process communication, or map a file directly into it. The munmap function releases this memory segment created by mmap. Their definitions are as follows:

1
2
3
#include <sys/mman.h>
void* mmap(void* start,size_t length,int prot,int flags,int fd,off_t offset);
int munmap(void* start,size_t length);

start参数允许用户使用某个特定的地址作为这段内存的起始地址。如果它被设置成NULL，则系统自动分配一个地址。length参数指定内存段的长度。prot参数用来设置内存段的访问权限。它可以取以下几个值的按位或：

  • PROT_READ，内存段可读。
  • PROT_WRITE，内存段可写。
  • PROT_EXEC, 内存段可执行。
  • PROT_NONE, 内存段不能被访问。

flags参数控制内存段内容被修改后程序的行为。（见书）

fd参数是被映射文件对应的文件描述符。它一般通过open系统调用获得。offset参数设置从文件的何处开始映射（对于不需要读入整个文件的情况）。

The start parameter allows the user to use a specific address as the starting address of this memory segment. If it is set to NULL, the system automatically allocates an address. The length parameter specifies the length of the memory segment. The prot parameter is used to set the access permissions of the memory segment. It can take the bitwise OR of the following values:

  • PROT_READ, the memory segment is readable.
  • PROT_WRITE, the memory segment is writable.
  • PROT_EXEC, the memory segment is executable.
  • PROT_NONE, the memory segment cannot be accessed.

The flags parameter controls the program’s behavior after the content of the memory segment is modified. (See the book.)

The fd parameter is the file descriptor of the mapped file. It is usually obtained through the open system call. The offset parameter sets where in the file to start mapping (for cases where the entire file does not need to be read in).

mmap函数成功时返回指向目标内存区域的指针，失败则返回MAP_FAILED（(void*)-1）并设置errno。munmap函数成功时返回0，失败则返回-1并设置errno。

On success, the mmap function returns a pointer to the target memory area; on failure it returns MAP_FAILED ((void*)-1) and sets errno. On success, the munmap function returns 0; on failure it returns -1 and sets errno.

splice函数

splice函数用于在两个文件描述符之间移动数据，也是零拷贝操作。splice函数的定义如下：

The splice Function

The splice function is used to move data between two file descriptors and is also a zero-copy operation. The definition of the splice function is as follows:

1
2
#include <fcntl.h>
ssize_t splice(int fd_in,loff_t* off_in,int fd_out,loff_t* off_out,size_t len,unsigned int flags);

fd_in参数是待输入数据的文件描述符。如果fd_in是一个管道文件描述符，那么off_in参数必须被设置为NULL。如果fd_in不是一个管道文件描述符（比如socket），那么off_in表示从输入数据流的何处开始读取数据。此时，若off_in被设置为NULL，则表示从输入数据流的当前偏移位置读入；若off_in不为NULL，则它将指出具体的偏移位置。fd_out/off_out参数的含义与fd_in/off_in相同，不过用于输出数据流。len参数指定移动数据的长度；flags参数则控制数据如何移动。

使用splice函数时，fd_in和fd_out必须至少有一个是管道文件描述符。splice函数调用成功时返回移动字节的数量。它可能返回0，表示没有数据需要移动，这发生在从管道中读取数据（fd_in是管道文件描述符）而该管道没有被写入任何数据时。splice函数失败时返回-1并设置errno。

The fd_in parameter is the file descriptor for input data. If fd_in is a pipe file descriptor, the off_in parameter must be set to NULL. If fd_in is not a pipe file descriptor (e.g., a socket), then off_in indicates where to start reading data from the input stream. In this case, if off_in is set to NULL, it means reading from the current offset position of the input stream; if off_in is not NULL, it specifies the specific offset position. The fd_out/off_out parameters have the same meaning as fd_in/off_in, but are used for the output stream. The len parameter specifies the length of data to move; the flags parameter controls how the data is moved.

When using the splice function, at least one of fd_in and fd_out must be a pipe file descriptor. On success, the splice function returns the number of bytes moved. It may return 0, meaning there is no data to move; this happens when reading from a pipe (fd_in is a pipe file descriptor) that has no data written to it. On failure, the splice function returns -1 and sets errno.

tee函数

tee函数在两个管道文件描述符之间复制数据，也是零拷贝操作。它不消耗数据，因此源文件描述符上的数据仍然可以用于后续的读操作。tee函数的原型如下：

The tee Function

The tee function copies data between two pipe file descriptors and is also a zero-copy operation. It does not consume data, so the data on the source file descriptor can still be used for subsequent read operations. The prototype of the tee function is as follows:

1
2
#include <fcntl.h>
ssize_t tee(int fd_in,int fd_out,size_t len,unsigned int flags);

该函数的参数的含义与splice相同（但fd_in和fd_out必须都是管道文件描述符）。tee函数成功时返回在两个文件描述符之间复制的数据数量（字节数）。返回0表示没有复制任何数据。tee失败时返回-1并设置errno。

The meanings of this function’s parameters are the same as splice (but both fd_in and fd_out must be pipe file descriptors). On success, the tee function returns the amount of data (in bytes) copied between the two file descriptors. Returning 0 means no data was copied. On failure, tee returns -1 and sets errno.

fcntl函数

fcntl函数，正如其名字（file control）描述的那样，提供了对文件描述符的各种控制操作。另外一个常见的控制文件描述符属性和行为的系统调用是ioctl，而且ioctl比fcntl能够执行更多的控制。但是，对于控制文件描述符常用的属性和行为，fcntl函数是由POSIX规范指定的首选方法。fcntl函数定义如下：

The fcntl Function

The fcntl function, as its name (file control) describes, provides various control operations on file descriptors. Another common system call for controlling file descriptor attributes and behavior is ioctl, and ioctl can perform more controls than fcntl. However, for common attributes and behaviors of file descriptors, fcntl is the preferred method specified by the POSIX standard. The fcntl function is defined as follows:

1
2
#include <fcntl.h>
int fcntl(int fd,int cmd,...);

fd参数是被操作的文件描述符，cmd参数指定执行何种类型的操作。根据操作类型的不同，该函数可能还需要第三个可选参数arg。

在网络编程中，fcntl函数通常用来将一个文件描述符设置为非阻塞的。

The fd parameter is the file descriptor to operate on, and the cmd parameter specifies the type of operation to perform. Depending on the operation type, the function may also need a third optional parameter arg.

In network programming, the fcntl function is usually used to set a file descriptor to non-blocking.

1
2
3
4
5
6
int setnonblocking(int fd){
    int old_option = fcntl(fd,F_GETFL); // 获取文件描述符旧的状态标志
    int new_option = old_option | O_NONBLOCK; // 设置非阻塞
    fcntl(fd,F_SETFL,new_option);
    return old_option; //返回旧的状态标志，以便日后恢复状态标志。
}

2022_3_6

2022_3_6-2

2022_3_6

2022_3_6-2