Starting with OpenVOS 17.1, STCP supports the time stamp, selective acknowledgment, and window scaling TCP options. This will allow for a more efficient data flow when TCP segments are dropped or when sending data over a high bandwidth delay connection (a connection over a link where the minimum bandwidth * overall latency > 65536). The effect of these changes is transparent to applications but there are a few things you should be aware of when using packet_monitor.
First, when STCP sends a connection request the TCP (SYN) segment will now contain an additional 16 bytes.
17:41:26.533 Xmit Ether Dst 00:23:54:52:18:6e Src 00:00:a8:41:34:56 Type 0800
+(IP)
IP Ver/HL 45, ToS 0, Len 3c, ID a789, Flg/Frg 0, TTL 3c, Prtl 6
Cksum f2f6, Src a4984dd9, Dst a4984d32
TCP from 164.152.77.217.55423 to 164.152.77.50.7777
seq 3825990103, ack n.a., window 8192, 20 data bytes, flags Syn.
X/Off 0a, Flags 02, Cksum b717, Urg-> 0000
offset 0 . . . 4 . . . 8 . . . C . . . 0...4... 8...C...
0 2 4 5 b4 3 3 6 4 2 8 a 32 2d bc 85 0 <<<4<<<< <<<2-<>
10 0 0 0 0
|
| 2 4 5 b4 | option 2 (minimum segment size), total length of option 4 bytes, value 05 b4 (1460 decimal) (these bytes are not new) |
| 3 3 6 | option 3 (window scale), total length of option 3 bytes, value 6 |
| 4 2 | option 4 (selective acknowledgment supported), total length 2 bytes |
| 8 a 32 2d bc 85 0 0 0 0 | option 8 (time stamp), total length 10 bytes, sending time 32 2d bc 85, return time 0 0 0 0 |
| 0 | option 0 (end of options) |
Second, the options sent in a connection reply (SYN/ACK) segment will depend on what the remote host’s SYN segment contains. STCP will include the selective acknowledgment supported option in its reply only if the connection request includes it. The reply will only include the window scale and time stamp options if the connection request also includes both options.
14:31:32.145 Rcvd Ether Dst 00:00:a8:42:34:56 Src 00:00:a8:42:52:22 Type 0800
+(IP)
IP Ver/HL 45, ToS 0, Len 3c, ID 44c, Flg/Frg 0, TTL 3c, Prtl 6
Cksum 95e6, Src a4984d80, Dst a4984dd9
TCP from 164.152.77.128.65007 to 164.152.77.217.7777
seq 3289184624, ack n.a., window 8192, 20 data bytes, flags Syn.
X/Off 0a, Flags 02, Cksum 91e8, Urg-> 0000
offset 0 . . . 4 . . . 8 . . . C . . . 0...4... 8...C...
0 2 4 5 b4 3 3 6 4 2 8 a 30 b8 95 1a 0 <<<4<<<< <<<08><
10 0 0 0 0 <<<<
14:31:32.146 Xmit Ether Dst 00:00:a8:42:52:22 Src 00:00:a8:42:34:56 Type 0800
+(IP)
IP Ver/HL 45, ToS 0, Len 3c, ID 0, Flg/Frg 0, TTL 3c, Prtl 6
Cksum 9a32, Src a4984dd9, Dst a4984d80
TCP from 164.152.77.217.7777 to 164.152.77.128.65007
seq 3172667591, ack 3289184625, window 8192, 20 data bytes, flags Syn Ack.
X/Off 0a, Flags 12, Cksum 6b81, Urg-> 0000
offset 0 . . . 4 . . . 8 . . . C . . . 0...4... 8...C...
0 2 4 5 b4 3 3 6 4 2 8 a 1 35 72 23 30 <<<4<<<< <<<<5r#0
10 b8 95 1a 0 8><<
|
By default Windows hosts only include the maximum segment size, window scale and selective acknowledgment supported options. Without the time stamp option in the connection request STCP’s reply will not include the time stamp or window scale options. Note that the 1’s included in the Window’s options are the NO-OP option and are used to make the number of option bytes an integer multiple of 4. The multiple end of options (0’s) at the end of the options in STCP’s reply are for the same reason.
14:54:02.635 Rcvd Ether Dst 00:00:a8:41:34:56 Src 00:23:54:52:18:6e Type 0800
+(IP)
IP Ver/HL 45, ToS 0, Len 34, ID a4f, Flg/Frg 4000, TTL 80, Prtl 6
Cksum 0c39, Src a4984d32, Dst a4984dd9
TCP from 164.152.77.50.13478 to 164.152.77.217.7777
seq 3499417963, ack n.a., window 8192, 12 data bytes, flags Syn.
X/Off 08, Flags 02, Cksum 65c8, Urg-> 0000
offset 0 . . . 4 . . . 8 . . . C . . . 0...4... 8...C...
0 2 4 5 b8 1 3 3 8 1 1 4 2 <<<8<<<< <<<<
14:54:02.636 Xmit Ether Dst 00:23:54:52:18:6e Src 00:00:a8:41:34:56 Type 0800
+(IP)
IP Ver/HL 45, ToS 0, Len 30, ID b3d5, Flg/Frg 0, TTL 3c, Prtl 6
Cksum e6b6, Src a4984dd9, Dst a4984d32
TCP from 164.152.77.217.7777 to 164.152.77.50.13478
seq 3986949958, ack 3499417964, window 8192, 8 data bytes, flags Syn Ack.
X/Off 07, Flags 12, Cksum 85e0, Urg-> 0000
offset 0 . . . 4 . . . 8 . . . C . . . 0...4... 8...C...
0 2 4 5 b4 4 2 0 0 <<<4<<
|
The confusing part comes after the connection is established. The following packet contains only 1 byte of data even though the data bytes field says 13. The first 12 bytes are two NO-OP options followed by a time stamp option. The data byte is the 0x at the end.
15:02:36.545 Rcvd Ether Dst 00:00:a8:41:34:56 Src 00:00:a8:42:52:22 Type 0800
+(IP)
IP Ver/HL 45, ToS 0, Len 35, ID 447, Flg/Frg 0, TTL 3c, Prtl 6
Cksum 95f2, Src a4984d80, Dst a4984dd9
TCP from 164.152.77.128.60312 to 164.152.77.217.7777
seq 2814493687, ack 4102208833, window 128, 13 data bytes, flags Push Ack
+.
X/Off 08, Flags 18, Cksum c9d0, Urg-> 0000
offset 0 . . . 4 . . . 8 . . . C . . . 0...4... 8...C...
0 1 1 8 a 2a 28 5d 8f 38 9f e0 a 0 <<<<*(]> 8>`<
|
How can I tell the difference between options and data? The X/Off value is the number of 32 bit words in the TCP Header. If there are no options, this value is 5 so a value of 8 indicates an addition of three 32 bit words or 12 more bytes.
This next packet is just an acknowledgment, it contains no data. Note that the offset is again 8 so there are 12 bytes of option and the data length is just 12.
15:02:39.758 Xmit Ether Dst 00:00:a8:42:52:22 Src 00:00:a8:41:34:56 Type 0800
+(IP)
IP Ver/HL 45, ToS 0, Len 34, ID 48a, Flg/Frg 0, TTL 3c, Prtl 6
Cksum 95b0, Src a4984dd9, Dst a4984d80
TCP from 164.152.77.217.7777 to 164.152.77.128.60312
seq 4102208833, ack 2814493688, window 128, 12 data bytes, flags Ack.
X/Off 08, Flags 10, Cksum a751, Urg-> 0000
offset 0 . . . 4 . . . 8 . . . C . . . 0...4... 8...C...
0 1 1 8 a 38 a0 2 91 2a 28 5d 8f <<<<8 <> *(]>
|
The following packet is an example of a segment containing a selective acknowledgment option with 3 blocks of acknowledgment numbers. Note the X/Off value of 0f indicating an additional 40 bytes ((15 – 5) * 4) in the TCP header which packet_monitor reports as data bytes.
9:52:39.491 Rcvd Ether Dst 00:00:a8:41:34:56 Src 00:04:96:52:21:6f Type 0800
+ (IP)
IP Ver/HL 45, ToS 60, Len 50, ID 59, Flg/Frg 0, TTL 35, Prtl 6
Cksum 4355, Src 866fc8b9, Dst a4984dd9
TCP from 134.111.200.185.ftp-data to 164.152.77.217.53828
seq 1652086450, ack 3779437719, window 4096, 40 data bytes, flags Ack.
X/Off 0f, Flags 10, Cksum c834, Urg-> 0000
offset 0 . . . 4 . . . 8 . . . C . . . 0...4... 8...C...
0 1 1 8 a 2 9e 86 bd 3c 4b dd 57 1 1 5 1a <<<<<<<< >K<W<<<<
10 e1 45 f6 77 e1 45 f8 83 e1 45 f2 5f e1 45 f4 6b <E<w<E<< <E<_<E<k
20 e1 45 c9 6f e1 45 cb 7b <E<o<E<{
|
The acknowledgment number in the TCP header is the right most edge of continuously received bytes. The data blocks in the selective acknowledgment option indicate blocks of data that have been received. The gaps between the blocks represent bytes that have not been received.
This would all be much simpler if packet_monitor correctly reported the options instead of reporting them as data. The bug stcp-2963 has been added to make packet_monitor do just that. The bug is scheduled to be fixed in a future release, until then you need to pay attention to the X/Off value.
The window scale option has the potential to be the most confusing. The following packet shows a receive window of only 4096 bytes.
9:52:38.975 Rcvd Ether Dst 00:00:a8:41:34:56 Src 00:04:96:52:21:6f Type 0800
+ (IP)
IP Ver/HL 45, ToS 60, Len 34, ID 8, Flg/Frg 0, TTL 35, Prtl 6
Cksum 43c2, Src 866fc8b9, Dst a4984dd9
TCP from 134.111.200.185.ftp-data to 164.152.77.217.53828
seq 1652086450, ack 3779392131, window 4096, 12 data bytes, flags Ack.
X/Off 08, Flags 10, Cksum a3cf, Urg-> 0000
offset 0 . . . 4 . . . 8 . . . C . . . 0...4... 8...C...
0 1 1 8 a 2 9e 86 39 3c 4b dc e3 <<<<<>><K<<<
|
This is an extremely small window but the size is misleading because window scaling is in effect. How can you tell? The only way to tell is to have observed the initial connection set up. In the following case you can see that both the SYN and the SYN-ACK segments include the window scaling option, and in both cases the scale value is 6 so the 4096 (2^12) window size in the above segment is shifted by 6 bits to give the actual window of 262144 (2^18 or 256K) bytes.
9:52:38.818 Rcvd Ether Dst 00:00:a8:41:34:56 Src 00:04:96:52:21:6f Type 0800
+ (IP)
IP Ver/HL 45, ToS 60, Len 3c, ID 5, Flg/Frg 0, TTL 35, Prtl 6
Cksum 43bd, Src 866fc8b9, Dst a4984dd9
TCP from 134.111.200.185.ftp-data to 164.152.77.217.53828
seq 1652086449, ack n.a., window 8192, 20 data bytes, flags Syn.
X/Off 0a, Flags 02, Cksum 29fd, Urg-> 0000
offset 0 . . . 4 . . . 8 . . . C . . . 0...4... 8...C...
0 2 4 2 18 3 3 6 4 2 8 a 2 9e 86 11 0 <<<<<<<< <<<<>><
10 0 0 0 0
9:52:38.819 Xmit Ether Dst 00:13:d4:59:7a:da Src 00:00:a8:41:34:56 Type 0800
+ (IP)
IP Ver/HL 45, ToS 0, Len 3c, ID a, Flg/Frg 0, TTL 3c, Prtl 6
Cksum 3d18, Src a4984dd9, Dst 866fc8b9
TCP from 164.152.77.217.53828 to 134.111.200.185.ftp-data
seq 3779391082, ack 1652086450, window 8192, 20 data bytes, flags Syn Ack
+.
X/Off 0a, Flags 12, Cksum 3b87, Urg-> 0000
offset 0 . . . 4 . . . 8 . . . C . . . 0...4... 8...C...
0 2 4 5 b4 3 3 6 4 2 8 a 3c 4b dc cf 2 <<<<<<<< <<<<<K<
10 9e 86 11 0 >><
|
These options are under the control of STCP parameters.
as: list_stcp_params
STCP Parameters:
. . . .
lowest IP max segment size [500-1480] (min_mss) 576
. . . .
maximum send window size [4096-1073725440] (max_send_ws) 1073725440
maximum huge windows [>=0] (max_huge_windows) 0
current huge windows 0
maximum 256k windows [>=0] (max_256k_windows) 25
current 256k windows 0
. . . . .
TCP RFC-1323 Policy [disable/allow/request] (tcp_rfc_1323_policy) request
TCP requested window scale [0-14] (tcp_window_scale) 6
TCP SACK policy [disable/allow/request] (tcp_SACK_policy) request
as:
|
The tcp_rfc_1323_policy option indicates whether time stamps will be used. The parameter values are disable/allow/request. The default of request indicates that the option is part of a connection request segment and if requested in a received connection request will be part of the reply segment. A value of allow indicates that the option will not be part of a connection request segment but may be part of a reply, if the received connection request includes it. The disable indicates the option will not be part of either type of segment. Setting the value to disable also indicates that the window scale option will not be part of the segment.
The tcp_window_scale option controls the scaling parameter. A zero value tells the remote host that window scaling is supported but not to scale STCP’s window.
15:48:43.409 Rcvd IP Ver/HL 45, ToS 0, Len 3c, ID 450, Flg/Frg 0, TTL 3
+c, Prtl 6
Cksum 95e2, Src a4984d80, Dst a4984dd9
TCP from 164.152.77.128.65410 to 164.152.77.217.7777
seq 1987362195, ack n.a., window 8192, 20 data bytes, flags Syn.
X/Off 0a, Flags 02, Cksum b8b3, Urg-> 0000
offset 0 . . . 4 . . . 8 . . . C . . . 0...4... 8...C...
0 2 4 5 b4 3 3 6 4 2 8 a 30 ca ac 69 0 <<<4<<<< <<<0J,i
10 0 0 0 0
15:48:48.575 Xmit IP Ver/HL 45, ToS 0, Len 3c, ID 2, Flg/Frg 0, TTL 3
+c, Prtl 6
Cksum 9a30, Src a4984dd9, Dst a4984d80
TCP from 164.152.77.217.7777 to 164.152.77.128.65410
seq 3853608469, ack 1987362196, window 8192, 20 data bytes, flags Syn Ack.
X/Off 0a, Flags 12, Cksum c550, Urg-> 0000
offset 0 . . . 4 . . . 8 . . . C . . . 0...4... 8...C...
0 2 4 5 b4 3 3 0 4 2 8 a 1 47 89 66 30 <<<4<< < <<<<G>f0
10 ca ac 69 0 J,i
|
The tcp_sack_policy controls the selective acknowledgment option. The values can be disable, allow, and request and mean exactly the same thing that they do for the tcp_rfc_1323_policy option.
Finally, with support of window scaling it is possible to have windows larger than the previous maximum of 64K , so two new window size pools have been created. These are reflected in the max_256k_windows and max_huge_windows pools. You can adjust the maximum number of sockets that can advertise a 256K window or an even larger window by setting the max_256k_windows or max_huge_windows values.
The max_send_ws parameter reflects the fact that the maximum possible remote host’s receive window is now 2^30 (1073725440 is actually 2^30 – 2^14). For a given socket the actual maximum send window size with be the minimum of max_send_ws parameter and the remote hosts advertised receive window size.