Linux 写入关闭的本地 TCP 套接字不会失败

Writing to a closed, local TCP socket not failing

提问by regularfry

I seem to be having a problem with my sockets. Below, you will see some code which forks a server and a client. The server opens a TCP socket, and the client connects to it and then closes it. Sleeps are used to coordinate the timing. After the client-side close(), the server tries to write() to its own end of the TCP connection. According to the write(2) man page, this shouldgive me a SIGPIPE and an EPIPE errno. However, I don't see this. From the server's point of view, the write to a local, closed socket succeeds, and absent the EPIPE I can't see how the server should be detecting that the client has closed the socket.

我的套接字似乎有问题。下面，您将看到一些分叉服务器和客户端的代码。服务器打开一个 TCP 套接字，客户端连接到它然后关闭它。睡眠用于协调时间。在客户端 close() 之后，服务器尝试 write() 到它自己的 TCP 连接端。根据 write(2) 手册页，这应该给我一个 SIGPIPE 和一个 EPIPE errno。但是，我没有看到这一点。从服务器的角度来看，写入本地关闭的套接字成功，如果没有 EPIPE，我看不到服务器应该如何检测客户端已关闭套接字。

In the gap between the client closing its end and the server attempting to write, a call to netstat will show that the connection is in a CLOSE_WAIT/FIN_WAIT2 state, so the server end should definitely be able to reject the write.

在客户端关闭其端和服务器尝试写入之间的间隙中，调用 netstat 将显示连接处于 CLOSE_WAIT/FIN_WAIT2 状态，因此服务器端肯定能够拒绝写入。

For reference, I'm on Debian Squeeze, uname -r is 2.6.39-bpo.2-amd64.

作为参考，我使用的是 Debian Squeeze，uname -r 是 2.6.39-bpo.2-amd64。

What's going on here?

这里发生了什么？

---

---

#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <sys/socket.h>
#include <sys/select.h>
#include <netinet/tcp.h>
#include <errno.h>
#include <string.h>
#include <stdlib.h>
#include <fcntl.h>

#include <netdb.h>

#define SERVER_ADDRESS "127.0.0.7"
#define SERVER_PORT 4777


#define myfail_if( test, msg ) do { if((test)){ fprintf(stderr, msg "\n"); exit(1); } } while (0)
#define myfail_unless( test, msg ) myfail_if( !(test), msg )

int connect_client( char *addr, int actual_port )
{
    int client_fd;

    struct addrinfo hint;
    struct addrinfo *ailist, *aip;


    memset( &hint, '
   EPIPE  fd is connected to a pipe or socket whose reading end is closed.
          When this happens the writing process will also receive  a  SIG-
          PIPE  signal.  (Thus, the write return value is seen only if the
          program catches, blocks or ignores this signal.)
', sizeof( struct addrinfo ) );
    hint.ai_socktype = SOCK_STREAM;

    myfail_if( getaddrinfo( addr, NULL, &hint, &ailist ) != 0, "getaddrinfo failed." );

    int connected = 0;
    for( aip = ailist; aip; aip = aip->ai_next ) {
        ((struct sockaddr_in *)aip->ai_addr)->sin_port = htons( actual_port );
        client_fd = socket( aip->ai_family, aip->ai_socktype, aip->ai_protocol );

        if( client_fd == -1) { continue; }
        if( connect( client_fd, aip->ai_addr, aip->ai_addrlen) == 0 ) {
            connected = 1;
            break;
        }
        close( client_fd );
    }

    freeaddrinfo( ailist );

    myfail_unless( connected, "Didn't connect." );
    return client_fd;
}


void client(){
    sleep(1);
    int client_fd = connect_client( SERVER_ADDRESS, SERVER_PORT );

    printf("Client closing its fd... ");
    myfail_unless( 0 == close( client_fd ), "close failed" );
    fprintf(stdout, "Client exiting.\n");
    exit(0);
}


int init_server( struct sockaddr * saddr, socklen_t saddr_len )
{
    int sock_fd;

    sock_fd = socket( saddr->sa_family, SOCK_STREAM, 0 );
    if ( sock_fd < 0 ){
        return sock_fd;
    }

    myfail_unless( bind( sock_fd, saddr, saddr_len ) == 0, "Failed to bind." );
    return sock_fd;
}

int start_server( const char * addr, int port )
{
    struct addrinfo *ailist, *aip;
    struct addrinfo hint;
    int sock_fd;

    memset( &hint, '
void test_socketpair () {
    int pair[2];
    socketpair(PF_LOCAL, SOCK_STREAM, 0, pair);
    close(pair[0]);
    if (send(pair[1], "a", 1, MSG_NOSIGNAL) < 0) perror("send");
}

void test_pipe () {
    int pair[2];
    pipe(pair);
    close(pair[0]);
    signal(SIGPIPE, SIG_IGN);
    if (write(pair[1], "a", 1) < 0) perror("send");
    signal(SIGPIPE, SIG_DFL);
}
', sizeof( struct addrinfo ) );
    hint.ai_socktype = SOCK_STREAM;
    myfail_if( getaddrinfo( addr, NULL, &hint, &ailist ) != 0, "getaddrinfo failed." );

    for( aip = ailist; aip; aip = aip->ai_next ){
        ((struct sockaddr_in *)aip->ai_addr)->sin_port = htons( port );
        sock_fd = init_server( aip->ai_addr, aip->ai_addrlen );
        if ( sock_fd > 0 ){
            break;
        } 
    }
    freeaddrinfo( aip );

    myfail_unless( listen( sock_fd, 2 ) == 0, "Failed to listen" );
    return sock_fd;
}


int server_accept( int server_fd )
{
    printf("Accepting\n");
    int client_fd = accept( server_fd, NULL, NULL );
    myfail_unless( client_fd > 0, "Failed to accept" );
    return client_fd;
}


void server() {
    int server_fd = start_server(SERVER_ADDRESS, SERVER_PORT);
    int client_fd = server_accept( server_fd );

    printf("Server sleeping\n");
    sleep(60);

    printf( "Errno before: %s\n", strerror( errno ) );
    printf( "Write result: %d\n", write( client_fd, "123", 3 ) );
    printf( "Errno after:  %s\n", strerror( errno ) );

    close( client_fd );
}


int main(void){
    pid_t clientpid;
    pid_t serverpid;

    clientpid = fork();

    if ( clientpid == 0 ) {
        client();
    } else {
        serverpid = fork();

        if ( serverpid == 0 ) {
            server();
        }
        else {
            int clientstatus;
            int serverstatus;

            waitpid( clientpid, &clientstatus, 0 );
            waitpid( serverpid, &serverstatus, 0 );

            printf( "Client status is %d, server status is %d\n", 
                    clientstatus, serverstatus );
        }
    }

    return 0;
}

采纳答案by jxh

This is what the Linux man page says about writeand EPIPE:

这就是 Linux 手册页所说的write和EPIPE：

int a_sock = socket(PF_INET, SOCK_STREAM, 0);
const int one = 1;
setsockopt(a_sock, SOL_SOCKET, SO_REUSEADDR, &one, sizeof(one));
struct sockaddr_in a_sin = {0};
a_sin.sin_port = htons(4321);
a_sin.sin_family = AF_INET;
a_sin.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
bind(a_sock, (struct sockaddr *)&a_sin, sizeof(a_sin));
listen(a_sock, 1);
int c_sock = socket(PF_INET, SOCK_STREAM, 0);
fcntl(c_sock, F_SETFL, fcntl(c_sock, F_GETFL, 0)|O_NONBLOCK);
connect(c_sock, (struct sockaddr *)&a_sin, sizeof(a_sin));
fcntl(c_sock, F_SETFL, fcntl(c_sock, F_GETFL, 0)&~O_NONBLOCK);
struct sockaddr_in s_sin = {0};
socklen_t s_sinlen = sizeof(s_sin);
int s_sock = accept(a_sock, (struct sockaddr *)&s_sin, &s_sinlen);
struct pollfd c_pfd = { c_sock, POLLOUT, 0 };
if (poll(&c_pfd, 1, -1) != 1) perror("poll");
int erropt = -1;
socklen_t errlen = sizeof(erropt);
getsockopt(c_sock, SOL_SOCKET, SO_ERROR, &erropt, &errlen);
if (erropt != 0) { errno = erropt; perror("connect"); }
puts("P|Recv-Q|Send-Q|Local Address|Foreign Address|State|");
char cmd[256];
snprintf(cmd, sizeof(cmd), "netstat -tn | grep ':%hu ' | sed 's/  */|/g'",
         ntohs(s_sin.sin_port));
puts("before close on client"); system(cmd);
close(c_sock);
puts("after close on client"); system(cmd);
if (send(s_sock, "a", 1, MSG_NOSIGNAL) < 0) perror("send");
puts("after send on server"); system(cmd);
puts("end of test");
sleep(5);

When Linux is using a pipeor a socketpair, it can and will check the reading endof the pair, as these two programs would demonstrate:

当 Linux 使用 apipe或 a 时socketpair，它可以并且将会检查对的读取端，这两个程序将演示：

P|Recv-Q|Send-Q|Local Address|Foreign Address|State|
before close on client
tcp|0|0|127.0.0.1:35790|127.0.0.1:4321|ESTABLISHED|
tcp|0|0|127.0.0.1:4321|127.0.0.1:35790|ESTABLISHED|
after close on client
tcp|0|0|127.0.0.1:35790|127.0.0.1:4321|FIN_WAIT2|
tcp|1|0|127.0.0.1:4321|127.0.0.1:35790|CLOSE_WAIT|
after send on server
end of test

Linux is able to do so, because the kernel has innate knowledge about the other end of the pipe or connected pair. However, when using connect, the state about the socket is maintained by the protocol stack. Your test demonstrates this behavior, but below is a program that does it all in a single thread, similar to the two tests above:

Linux 能够这样做，因为内核具有关于管道或连接对的另一端的先天知识。但是，在使用时connect，套接字的状态由协议栈维护。您的测试演示了这种行为，但下面是一个在单个线程中完成所有操作的程序，类似于上面的两个测试：

16:45:28 127.0.0.1 > 127.0.0.1
 .809578 IP .35790 > .4321: S 1062313174:1062313174(0) win 32792 <mss 16396,sackOK,timestamp 3915671437 0,nop,wscale 7>
 .809715 IP .4321 > .35790: S 1068622806:1068622806(0) ack 1062313175 win 32768 <mss 16396,sackOK,timestamp 3915671437 3915671437,nop,wscale 7>
 .809583 IP .35790 > .4321: . ack 1 win 257 <nop,nop,timestamp 3915671437 3915671437>
 .840364 IP .35790 > .4321: F 1:1(0) ack 1 win 257 <nop,nop,timestamp 3915671468 3915671437>
 .841170 IP .4321 > .35790: . ack 2 win 256 <nop,nop,timestamp 3915671469 3915671468>
 .865792 IP .4321 > .35790: P 1:2(1) ack 2 win 256 <nop,nop,timestamp 3915671493 3915671468>
 .865809 IP .35790 > .4321: R 1062313176:1062313176(0) win 0

If you run the above program, you will get output similar to this:

如果你运行上面的程序，你会得到类似这样的输出：

//close(c_sock);
shutdown(c_sock, SHUT_WR);

This shows it took one writefor the sockets to transition to the CLOSEDstates. To find out why this occurred, a TCP dump of the transaction can be useful:

这表明write套接字转换到CLOSED状态花了一个时间。要找出发生这种情况的原因，事务的 TCP 转储可能很有用：

P|Recv-Q|Send-Q|Local Address|Foreign Address|State|
before close on client
tcp|0|0|127.0.0.1:4321|127.0.0.1:56355|ESTABLISHED|
tcp|0|0|127.0.0.1:56355|127.0.0.1:4321|ESTABLISHED|
after close on client
tcp|1|0|127.0.0.1:4321|127.0.0.1:56355|CLOSE_WAIT|
tcp|0|0|127.0.0.1:56355|127.0.0.1:4321|FIN_WAIT2|
after send on server
tcp|1|0|127.0.0.1:4321|127.0.0.1:56355|CLOSE_WAIT|
tcp|1|0|127.0.0.1:56355|127.0.0.1:4321|FIN_WAIT2|
end of test

The first three lines represent the 3-way handshake. The fourth line is the FINpacket the client sends to the server, and the fifth line is the ACKfrom the server, acknowledging receipt. The sixth line is the server trying to send 1 byte of data to the client with the PUSHflag set. The final line is the client RESETpacket, which causes the TCP state for the connection to be freed, and is why the third netstatcommand did not result in any output in the test above.

前三行代表 3 次握手。第四行是FIN客户端发送给服务器的数据包，第五行是ACK来自服务器的确认收到的数据包。第六行是服务器尝试向PUSH设置了标志的客户端发送 1 个字节的数据。最后一行是客户端RESET数据包，它导致连接的 TCP 状态被释放，这也是为什么第三个netstat命令在上面的测试中没有产生任何输出的原因。

So, the server doesn't know the client will reset the connection until after it tries to send some data to it. The reason for the reset is because the client called close, instead of something else.

因此，服务器不知道客户端会在尝试向其发送一些数据之前重置连接。重置的原因是因为客户端调用了close，而不是其他东西。

The server cannot know for certain what system call the client has actually issued, it can only follow the TCP state. For example, we could replace the closecall with a call to shutdowninstead.

服务器无法确定客户端实际发出了什么系统调用，它只能遵循 TCP 状态。例如，我们可以用close调用来shutdown代替调用。

17:09:18 127.0.0.1 > 127.0.0.1
 .722520 IP .56355 > .4321: S 2558095134:2558095134(0) win 32792 <mss 16396,sackOK,timestamp 3917101399 0,nop,wscale 7>
 .722594 IP .4321 > .56355: S 2563862019:2563862019(0) ack 2558095135 win 32768 <mss 16396,sackOK,timestamp 3917101399 3917101399,nop,wscale 7>
 .722615 IP .56355 > .4321: . ack 1 win 257 <nop,nop,timestamp 3917101399 3917101399>
 .748838 IP .56355 > .4321: F 1:1(0) ack 1 win 257 <nop,nop,timestamp 3917101425 3917101399>
 .748956 IP .4321 > .56355: . ack 2 win 256 <nop,nop,timestamp 3917101426 3917101425>
 .764894 IP .4321 > .56355: P 1:2(1) ack 2 win 256 <nop,nop,timestamp 3917101442 3917101425>
 .764903 IP .56355 > .4321: . ack 2 win 257 <nop,nop,timestamp 3917101442 3917101442>
17:09:23
 .786921 IP .56355 > .4321: R 2:2(0) ack 2 win 257 <nop,nop,timestamp 3917106464 3917101442>

The difference between shutdownand closeis that shutdownonly governs the state of the connection, while closealso governs the state of the file descriptorthat represents the socket. A shutdownwill not closea socket.

shutdown和之间的区别close是shutdown只管理连接的状态，同时close也管理代表套接字的文件描述符的状态。Ashutdown不会close是套接字。

The output will be different with the shutdownchange:

输出将随着shutdown更改而不同：

struct pollfd s_pfd = { s_sock, POLLIN|POLLOUT, 0 };
if (poll(&s_pfd, 1, -1) != 1) perror("poll");
if (s_pfd.revents|POLLIN) {
    char c;
    int r;
    while ((r = recv(s_sock, &c, 1, MSG_DONTWAIT)) == 1) {}
    if (r == 0) { /*...FIN received...*/ }
    else if (errno == EAGAIN) { /*...no more data to read for now...*/ }
    else { /*...some other error...*/ perror("recv"); }
}

The TCP dump will show also show something different:

TCP 转储还将显示一些不同的内容：

shutdown(s_sock, SHUT_WR);
if (send(s_sock, "a", 1, MSG_NOSIGNAL) < 0) perror("send");

Notice the reset at the end comes 5 seconds after the last ACKpacket. This reset is due to the program shutting down without properly closing the sockets. It is the ACKpacket from the client to the server before the reset that is different than before. This is the indication that the client did not use close. In TCP, the FINindication is really an indication that there is no more data to be sent. But since a TCP connection is bi-directional, the server that receives the FINassumes the client can still receive data. In the case above, the client in fact does accept the data.

请注意，在最后一个ACK数据包之后 5 秒，最后的重置发生了。此重置是由于程序在没有正确关闭套接字的情况下关闭。ACK与之前不同的是重置前从客户端到服务器的数据包。这表明客户端没有使用close. 在 TCP 中，该FIN指示实际上是没有更多数据要发送的指示。但是由于 TCP 连接是双向的，接收到的服务器FIN假定客户端仍然可以接收数据。在上述情况下，客户端实际上确实接受了数据。

Whether the client uses closeor SHUT_WRto issue a FIN, in either case you can detect the arrival of the FINby polling on the server socket for a readable event. If after calling readthe result is 0, then you know the FINhas arrived, and you can do what you wish with that information.

无论客户端使用close还是SHUT_WR发出FIN，在任何一种情况下，您都可以FIN通过在服务器套接字上轮询可读事件来检测 的到达。如果在调用read结果之后是0，那么您知道FIN已经到达，并且您可以使用该信息做您想做的事情。

struct linger lo = { 1, 0 };
setsockopt(c_sock, SOL_SOCKET, SO_LINGER, &lo, sizeof(lo));
close(c_sock);

Now, it is trivially true that if the server issues SHUT_WRwith shutdownbefore it tries to do a write, it will in fact get the EPIPEerror.

现在，它是平凡的事实，如果服务器的问题SHUT_WR与shutdown它试图做一个写之前，它实际上将得到EPIPE错误。

P|Recv-Q|Send-Q|Local Address|Foreign Address|State|
before close on client
tcp|0|0|127.0.0.1:35043|127.0.0.1:4321|ESTABLISHED|
tcp|0|0|127.0.0.1:4321|127.0.0.1:35043|ESTABLISHED|
after close on client
send: Connection reset by peer
after send on server
end of test

If, instead, you want the client to indicate an immediate reset to the server, you can force that to happen on most TCP stacks by enabling the linger option, with a linger timeout of 0prior to calling close.

相反，如果您希望客户端指示立即重置服务器，您可以通过启用 linger 选项强制在大多数 TCP 堆栈上发生这种情况0，调用之前的 linger 超时为close。

17:44:21 127.0.0.1 > 127.0.0.1
 .662163 IP .35043 > .4321: S 498617888:498617888(0) win 32792 <mss 16396,sackOK,timestamp 3919204411 0,nop,wscale 7>
 .662176 IP .4321 > .35043: S 497680435:497680435(0) ack 498617889 win 32768 <mss 16396,sackOK,timestamp 3919204411 3919204411,nop,wscale 7>
 .662184 IP .35043 > .4321: . ack 1 win 257 <nop,nop,timestamp 3919204411 3919204411>
 .691207 IP .35043 > .4321: R 1:1(0) ack 1 win 257 <nop,nop,timestamp 3919204440 3919204411>

With the above change, the output of the program becomes:

有了上面的改动，程序的输出变成了：

close(client_fd);
printf( "Write result: %d\n", write( client_fd, "123", 3 ) );

The sendgets an immediate error in this case, but it is not EPIPE, it is ECONNRESET. The TCP dump reflects this as well:

在send这种情况下获取的即时错误，但它不是EPIPE，它是ECONNRESET。TCP 转储也反映了这一点：

...
printf( "Errno before: %s\n", strerror( errno ) );
printf( "Write result: %d\n", write( client_fd, "123", 3 ) );
printf( "Errno after:  %s\n", strerror( errno ) );

printf( "Errno before: %s\n", strerror( errno ) );
printf( "Write result: %d\n", write( client_fd, "A", 1 ) );
printf( "Errno after:  %s\n", strerror( errno ) );
...

The RESETpacket comes right after the 3-way handshake completes. However, using this option has its dangers. If the other end has unread data in the socket buffer when the RESETarrives, that data will be purged, causing the data to be lost. Forcing a RESETto be sent is usually used in request/response style protocols. The sender of the request can know there can be no data lost when it receives the entire response to its request. Then, it is safe for the request sender to force a RESETto be sent on the connection.

该RESET数据包中的3次握手完成后马上来了。但是，使用此选项有其危险。如果另一端RESET到达时socket缓冲区中有未读数据，则该数据将被清除，导致数据丢失。强制RESET发送a通常用于请求/响应风格的协议中。请求的发送方在收到对其请求的整个响应时，可以知道不会丢失任何数据。然后，请求发送方强制RESET在连接上发送 a 是安全的。

回答by Eric Y

I suspect that what's happening is the server side socket is still valid so your write call is making a valid attempt at writing to your file descriptor even though your TCP session is in a closed state. If I am completely wrong let me know.

我怀疑发生的事情是服务器端套接字仍然有效，因此即使您的 TCP 会话处于关闭状态，您的 write 调用也会有效地尝试写入您的文件描述符。如果我完全错了，请告诉我。

回答by David G

I guess that you're running into the TCP stack detecting a failed send and attempting retransmission. Do subsequent calls to write()fail silently? In other words, try writing five times to the closed socket and see if you eventually get a SIGPIPE. And when you say the write 'succeeds', do you get a return result of 3?

我猜您遇到了 TCP 堆栈，检测到发送失败并尝试重新传输。后续调用是否write()会静默失败？换句话说，尝试向关闭的套接字写入五次，看看是否最终获得了 SIGPIPE。当你说写'succeeds'时，你得到的返回结果是3吗？

回答by Tanmoy Bandyopadhyay

You have two sockets - one for the client and another for the server. Now your client is doing the active close.This means TCP's conection termination has been started by the client ( A tcp FIN segment has been sent from the client send).

您有两个套接字 - 一个用于客户端，另一个用于服务器。现在您的客户端正在执行主动关闭。这意味着客户端已启动 TCP 的连接终止（客户端发送的 tcp FIN 段已发送）。

At this stage you see the client socket in FIN_WAIT1 state. Now what is the state of the server socket now? It is in CLOSE_WAIT state.So the server socket is not closed.

在此阶段，您会看到客户端套接字处于 FIN_WAIT1 状态。现在服务器套接字的状态是什么？它处于 CLOSE_WAIT 状态。所以服务器套接字没有关闭。

The FIN from the server has not been sent yet. (Why - since the application has not closed the socket). At this stage you are writing over the server socket so you are not getting an error.

来自服务器的 FIN 尚未发送。（为什么 - 因为应用程序还没有关闭套接字）。在此阶段，您正在写入服务器套接字，因此您不会收到错误消息。

Now if you want to see the error just write close(client_fd) before writing over the socket.

现在，如果您想查看错误，只需在写入套接字之前写入 close(client_fd) 即可。

Accepting
Server sleeping
Client closing its fd... Client exiting.
Errno before: Success
Write result: 3
Errno after:  Success
Errno before: Success
Client status is 0, server status is 13

Here the server socket is no more in CLOSE_WAIT state so you can see return value of write is -ve to indicate the error. I hope this clarifies.

这里服务器套接字不再处于 CLOSE_WAIT 状态，因此您可以看到 write 的返回值是 -ve 以指示错误。我希望这能澄清。

回答by alk

After having called write()one (first) time (as coded in your example) after the client close()ed the socket, you'll be getting the expected EPIPEand SIGPIPEon any successive call to write().

在write()客户端调用close()套接字后调用一次（第一次）（如您的示例中的编码）之后，您将获得预期的EPIPE和SIGPIPE对 write() 的任何连续调用。

Just try adding another write() to provoke the error:

只需尝试添加另一个 write() 来引发错误：

##代码##

The output will be:

输出将是：

##代码##

The output of the last two printf()s is missing as the process terminates due to SIGPIPEbeing raised by the second call to write(). To avoid the termination of the process, you might like to make the process ignore SIGPIPE.

printf()由于SIGPIPE第二次调用引发进程终止，最后两个s的输出丢失write()。为避免进程终止，您可能希望进程忽略SIGPIPE。