Wireshark & Packetdrill | TCP Thin Stream 重传 - 新鲜讯息

无用的知识又增加了

实验目的

基于 packetdrill TCP 三次握手脚本，通过构造模拟服务器端场景，研究测试 TCP thin stream 下的重传行为。

基础脚本

# cat tcp_thin_stream_000.pkt 0   socket(..., SOCK_STREAM, IPPROTO_TCP) = 3+0  setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0+0  bind(3, ..., ...) = 0+0  listen(3, 1) = 0+0 < S 0:0(0) win 10000 <mss 1000>+0 > S. 0:0(0) ack 1 <...>+0.01 < . 1:1(0) ack 1 win 10000+0 accept(3, ..., ...) = 4#

TCP Thin Stream

TCP Thin Stream 指数据传输速率低于 TCP ACK 时钟机制最低要求的数据流，典型特征为低速率、小数据包、长周期传输。具体来说，在 Thin Stream 场景下，由于应用层数据产生周期大于网络的往返时延（RTT），发送方即使在接收到 ACK 后，也往往没有新的应用数据可供发送，导致 TCP 处于“等待应用数据”和“等待 ACK”的双重空闲状态。因此，这种流量的有效发送速率不仅仅由应用层数据产生速率决定，还会受到 RTT、滑动窗口大小、Nagle 算法及延迟确认等机制的叠加影响。

为了改善 thin stream 的传输，Linux 曾引入了两个控制参数：tcp_thin_dupack 和 tcp_thin_linear_timeouts，而在较新内核中只保留了后者，默认关闭，在该参数打开的时候，前 6 次 RTO 超时触发的重传并不进行指数回退。

# sysctl -a|grep thinnet.ipv4.tcp_thin_linear_timeouts = 0#

Linux 中会判断如果当前没有处于初始的慢启动过程并且已经发出去的还没有收到 ACK 的数据包的个数小于 4 时，则会认定为 thin stream。

/* Determines whether this is a thin stream (which may suffer from * increased latency). Used to trigger latency-reducing mechanisms. */staticinlinebooltcp_stream_is_thin(struct tcp_sock *tp){return tp->packets_out < 4 && !tcp_in_initial_slowstart(tp);}#define TCP_INFINITE_SSTHRESH0x7fffffffstaticinlinebooltcp_in_initial_slowstart(conststruct tcp_sock *tp){return tp->snd_ssthresh >= TCP_INFINITE_SSTHRESH;}

基础测试

首先模拟超时重传现象，观察数据发送端的重传行为，脚本如下。

# cat tcp_thin_stream_001.pkt 0   socket(..., SOCK_STREAM, IPPROTO_TCP) = 3+0  setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0+0  setsockopt(3, SOL_TCP, TCP_NODELAY, [1], 4) = 0+0  bind(3, ..., ...) = 0+0  listen(3, 1) = 0+0  < S 0:0(0) win 10000 <mss 1460,nop,nop,sackOK>+0  > S. 0:0(0) ack 1 <...>+0.01 < . 1:1(0) ack 1 win 10000+0 accept(3, ..., ...) = 4+0 write(4, ..., 100) = 100+0.3 < . 1:1(0) ack 101 win 10000+0 write(4, ..., 100) = 100+0.01 < . 1:1(0) ack 201 win 10000+0 write(4, ..., 100) = 100+0 `sleep 10`#

通过 tcpdump 捕获数据包如下，发送端发出 Seq 201:301 数据包时，因未得到 ACK 确认，此时发生了超时重传和指数回退。

# packetdrill tcp_thin_stream_001.pkt ## tcpdump -i any -nn port 8080tcpdump: data link type LINUX_SLL2tcpdump: verbose output suppressed, use -v[v]... for full protocol decodelistening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes17:10:59.646993 tun0  In  IP 192.0.2.1.34395 > 192.168.162.123.8080: Flags [S], seq 0, win 10000, options [mss 1460,nop,nop,sackOK], length 017:10:59.647021 tun0  Out IP 192.168.162.123.8080 > 192.0.2.1.34395: Flags [S.], seq 787582622, ack 1, win 64240, options [mss 1460,nop,nop,sackOK], length 017:10:59.657112 tun0  In  IP 192.0.2.1.34395 > 192.168.162.123.8080: Flags [.], ack 1, win 10000, length 017:10:59.657201 tun0  Out IP 192.168.162.123.8080 > 192.0.2.1.34395: Flags [P.], seq 1:101, ack 1, win 64240, length 100: HTTP17:10:59.870660 tun0  Out IP 192.168.162.123.8080 > 192.0.2.1.34395: Flags [P.], seq 1:101, ack 1, win 64240, length 100: HTTP17:10:59.957231 tun0  In  IP 192.0.2.1.34395 > 192.168.162.123.8080: Flags [.], ack 101, win 10000, length 017:10:59.957280 tun0  Out IP 192.168.162.123.8080 > 192.0.2.1.34395: Flags [P.], seq 101:201, ack 1, win 64240, length 100: HTTP17:10:59.967358 tun0  In  IP 192.0.2.1.34395 > 192.168.162.123.8080: Flags [.], ack 201, win 10000, length 017:10:59.967431 tun0  Out IP 192.168.162.123.8080 > 192.0.2.1.34395: Flags [P.], seq 201:301, ack 1, win 64240, length 100: HTTP17:11:00.182659 tun0  Out IP 192.168.162.123.8080 > 192.0.2.1.34395: Flags [P.], seq 201:301, ack 1, win 64240, length 100: HTTP17:11:00.622664 tun0  Out IP 192.168.162.123.8080 > 192.0.2.1.34395: Flags [P.], seq 201:301, ack 1, win 64240, length 100: HTTP17:11:01.486672 tun0  Out IP 192.168.162.123.8080 > 192.0.2.1.34395: Flags [P.], seq 201:301, ack 1, win 64240, length 100: HTTP17:11:03.214703 tun0  Out IP 192.168.162.123.8080 > 192.0.2.1.34395: Flags [P.], seq 201:301, ack 1, win 64240, length 100: HTTP17:11:06.638670 tun0  Out IP 192.168.162.123.8080 > 192.0.2.1.34395: Flags [P.], seq 201:301, ack 1, win 64240, length 100: HTTP17:11:10.001870 ?     Out IP 192.168.162.123.8080 > 192.0.2.1.34395: Flags [F.], seq 301, ack 1, win 64240, length 017:11:10.001909 ?     In  IP 192.0.2.1.34395 > 192.168.162.123.8080: Flags [R.], seq 1, ack 201, win 10000, length 0#

修改 tcp_thin_linear_timeouts 值为 1 。

# sysctl -a|grep thinnet.ipv4.tcp_thin_linear_timeouts = 0# sysctl -q net.ipv4.tcp_thin_linear_timeouts=1# sysctl -a|grep thinnet.ipv4.tcp_thin_linear_timeouts = 1#

继续运行上述脚本，通过 tcpdump 捕获数据包如下，同样发送端发出 Seq 201:301 数据包时，因未得到 ACK 确认发生了超时重传，但相较上一次实验，前 6 次并未发生指数回退，一直是 216ms 左右，而在第 7 次 RTO 超时重传之后才开始指数回退，不断翻倍。

# packetdrill tcp_thin_stream_001.pkt ## tcpdump -i any -nn port 8080tcpdump: data link type LINUX_SLL2tcpdump: verbose output suppressed, use -v[v]... for full protocol decodelistening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes17:15:46.666970 tun0  In  IP 192.0.2.1.58771 > 192.168.136.16.8080: Flags [S], seq 0, win 10000, options [mss 1460,nop,nop,sackOK], length 017:15:46.666997 tun0  Out IP 192.168.136.16.8080 > 192.0.2.1.58771: Flags [S.], seq 924761446, ack 1, win 64240, options [mss 1460,nop,nop,sackOK], length 017:15:46.677120 tun0  In  IP 192.0.2.1.58771 > 192.168.136.16.8080: Flags [.], ack 1, win 10000, length 017:15:46.677307 tun0  Out IP 192.168.136.16.8080 > 192.0.2.1.58771: Flags [P.], seq 1:101, ack 1, win 64240, length 100: HTTP17:15:46.890660 tun0  Out IP 192.168.136.16.8080 > 192.0.2.1.58771: Flags [P.], seq 1:101, ack 1, win 64240, length 100: HTTP17:15:46.977340 tun0  In  IP 192.0.2.1.58771 > 192.168.136.16.8080: Flags [.], ack 101, win 10000, length 017:15:46.977373 tun0  Out IP 192.168.136.16.8080 > 192.0.2.1.58771: Flags [P.], seq 101:201, ack 1, win 64240, length 100: HTTP17:15:46.987501 tun0  In  IP 192.0.2.1.58771 > 192.168.136.16.8080: Flags [.], ack 201, win 10000, length 017:15:46.987616 tun0  Out IP 192.168.136.16.8080 > 192.0.2.1.58771: Flags [P.], seq 201:301, ack 1, win 64240, length 100: HTTP17:15:47.206682 tun0  Out IP 192.168.136.16.8080 > 192.0.2.1.58771: Flags [P.], seq 201:301, ack 1, win 64240, length 100: HTTP17:15:47.422671 tun0  Out IP 192.168.136.16.8080 > 192.0.2.1.58771: Flags [P.], seq 201:301, ack 1, win 64240, length 100: HTTP17:15:47.638665 tun0  Out IP 192.168.136.16.8080 > 192.0.2.1.58771: Flags [P.], seq 201:301, ack 1, win 64240, length 100: HTTP17:15:47.854666 tun0  Out IP 192.168.136.16.8080 > 192.0.2.1.58771: Flags [P.], seq 201:301, ack 1, win 64240, length 100: HTTP17:15:48.070668 tun0  Out IP 192.168.136.16.8080 > 192.0.2.1.58771: Flags [P.], seq 201:301, ack 1, win 64240, length 100: HTTP17:15:48.286663 tun0  Out IP 192.168.136.16.8080 > 192.0.2.1.58771: Flags [P.], seq 201:301, ack 1, win 64240, length 100: HTTP17:15:48.502663 tun0  Out IP 192.168.136.16.8080 > 192.0.2.1.58771: Flags [P.], seq 201:301, ack 1, win 64240, length 100: HTTP17:15:48.942664 tun0  Out IP 192.168.136.16.8080 > 192.0.2.1.58771: Flags [P.], seq 201:301, ack 1, win 64240, length 100: HTTP17:15:49.806695 tun0  Out IP 192.168.136.16.8080 > 192.0.2.1.58771: Flags [P.], seq 201:301, ack 1, win 64240, length 100: HTTP17:15:51.534675 tun0  Out IP 192.168.136.16.8080 > 192.0.2.1.58771: Flags [P.], seq 201:301, ack 1, win 64240, length 100: HTTP17:15:55.150705 tun0  Out IP 192.168.136.16.8080 > 192.0.2.1.58771: Flags [P.], seq 201:301, ack 1, win 64240, length 100: HTTP17:15:56.991356 ?     Out IP 192.168.136.16.8080 > 192.0.2.1.58771: Flags [F.], seq 301, ack 1, win 64240, length 017:15:56.991380 ?     In  IP 192.0.2.1.58771 > 192.168.136.16.8080: Flags [R.], seq 1, ack 201, win 10000, length 0#