First a little background, TCP has a concept called maximum segment size (MSS). This is the largest segment (of data) that the TCP stack will accept. It is advertized to the remote peer in the SYN segment when the connection is established. The following packet decode shows the SYN segment with the TCP MSS option highlighted. All TCP options have the same format, 1 byte for the option number, 2 in this case, 1 byte for the total length of the option (4) and the option’s value, in this case 5b4 hex or 1460 decimal.
packet_monitor -interface #sdlmux.m16.11-3 -hex_dump -numeric -time_stamp -verbo +se -pkt_hdr -filter -host 10.64.77.50 dir icmp type + tcp hh:mm:ss.ttt dir len proto source destination src port ds +t port type 10:11:56.751 Xmit Ether Dst 00:23:54:52:18:6e Src 00:00:a8:43:52:22 Type 0800 +(IP) IP Ver/HL 45, ToS 0, Len 2c, ID be0b, Flg/Frg 0, TTL 3c, Prtl 6 Cksum dcdd, Src a404d80, Dst a404d32 TCP from 10.64.77.128.55911 to 10.64.77.50.telnet seq 1305537174, ack n.a., window 8192, 4 data bytes, flags Syn. X/Off 06, Flags 02, Cksum 815d, Urg-> 0000 offset 0 . . . 4 . . . 8 . . . C . . . 0...4...8...C... 0 2 4 5 b4 <<<4
For a connection made to a host on a local network, STCP uses a value of 1460. Once you add in all the protocol overhead you end up with an Ethernet frame of 1518 bytes – the maximum Ethernet frame size. For connections made to hosts on non-local segments STCP uses a value derived from the min_mss parameter. Using the default min_mss value will result in an MSS value of 536. This value is the minimum size that all IP based network equipment, i.e. routers must be willing to forward without IP fragmentation, the specification that established 536 as a minimum was written in 1983 in RFC 879 – “The TCP Maximum Segment Size and related Topics”.
Today it is probable that all of your routers can handle a larger size. You can therefore raise the STCP minimum from 536 to something larger. How much larger is the key question. The danger in making it too large is fragmentation. The worst scenario, but the easiest to diagnose, is that the router will choose to discard the TCP segment instead of fragmenting it. In this case you will be able to establish a connection but when transferring data the connection will fail. A trace on the sending host will show multiple retransmissions of large TCP segments until the connection times out. The trace may also show ICMP destination unreachable messages from the router that is discarding the segment. If the router chooses to fragment the segment it will probably give the fragmentation process a low priority, this may actually result in a reduction of throughput. Also the probability of losing the segment due to corruption or congestion goes up with the number of fragments. The easiest way to tell if this is the case is by running a packet_monitor trace on the module. If the TCP segments come in as fragments you know that fragmentation has occurred.
The following three Ethernet frames represent one TCP segment fragmented into three parts. You can tell the first frame is a fragment because the Flg/Frg value is 2000. The 2 indicates that the More Fragments bit is set and the 000 is the fragment offset, in this case 0 indicates it is the first fragment. Packet monitor actually flags the next 2 frames as fragments. The Flg/Frg value in frame 2 is 2048, again the 2 indicates More Fragments bit is set and the 48 is the offset (in multiples of 8 bytes) of the data in the original IP datagram. The third frame has a Flg/Frag of just 90, or more accurately 0090. Since the More Fragments bit is 0 we know that this is the last fragment. We know that all these frames belong together because they all have the same IP ID value, 2e87. The data length in the first frame is an indication of the MSS value that should be set, in this case 556.
15:52:27.081 Rcvd Ether Dst 00:00:a8:43:52:22 Src 00:12:3f:82:57:10 Type 0800 +(IP) IP Ver/HL 45, ToS 8, Len 254, ID 2e87, Flg/Frg 2000, TTL 3f, Prtl 6 Cksum 1422, Src c0a86432, Dst a404d80 TCP from 192.168.100.50.32781 to 10.64.77.128.ftp-data seq 568258166, ack 840605671, window 5440, 556 data bytes, flags Ack. X/Off 05, Flags 10, Cksum d149, Urg-> 0000 offset 0 . . . 4 . . . 8 . . . C . . . 0…4… 8…C… 0 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 * 5678901234567890 10 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 * 1234567890123456 20 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 * 7890123456789012 30 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 * 3456789012345678 40 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 * 9012345678901234 50 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 * 5678901234567890 60 d a 31 32 33 34 35 36 37 38 39 30 31 32 33 34 * <<12345678901234 70 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 * 5678901234567890
15:52:33.793 Rcvd Ether Dst 00:00:a8:43:52:22 Src 00:12:3f:82:57:10 Type 0800 +(IP) IP Ver/HL 45, ToS 8, Len 254, ID 2e87, Flg/Frg 2048, TTL 3f, Prtl 6 Cksum 13da, Src c0a86432, Dst a404d80 TCP from 192.168.100.50. to 10.64.77.128. *fragments*, 576 bytes data offset 0 . . . 4 . . . 8 . . . C . . . 0…4… 8…C… 0 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 * 1234567890123456 10 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 * 7890123456789012 20 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 * 3456789012345678 30 39 30 d a 31 32 33 34 35 36 37 38 39 30 31 32 * 90<<123456789012 40 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 * 3456789012345678 50 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 * 9012345678901234 60 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 * 5678901234567890 70 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 * 1234567890123456
15:52:33.793 Rcvd Ether Dst 00:00:a8:43:52:22 Src 00:12:3f:82:57:10 Type 0800 +(IP) IP Ver/HL 45, ToS 8, Len f8, ID 2e87, Flg/Frg 90, TTL 3f, Prtl 6 Cksum 34ee, Src c0a86432, Dst a404d80 TCP from 192.168.100.50. to 10.64.77.128. *fragments., 228 bytes data offset 0 . . . 4 . . . 8 . . . C . . . 0…4… 8…C… 0 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 * 5678901234567890 10 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 * 1234567890123456 20 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 * 7890123456789012 30 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 * 3456789012345678 40 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 * 9012345678901234 50 35 36 37 38 39 30 d a 31 32 33 34 35 36 37 38 * 567890<<12345678 60 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 * 9012345678901234 70 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 * 5678901234567890
The problem with increasing the MSS value is that it increases it for all TCP connections. You cannot increase it for just one or a few destinations. Therefore you need to carefully test this change to gauge its effect on all your connections.
You can see the current MSS value and change it with the following two commands:
analyze_system -request_line 'list_stcp_params min_mss' -quitOpenVOS Release 17.0.1as, analyze_system Release 17.0.1asCurrent process is 357, ptep 8FA86000, Noah_Davids.SysAdmin
maximum tcp segment size [500-1480] (min_mss) 556
ready 10:52:07analyze_system -request_line 'set_stcp_param min_mss 1480' -quit OpenVOS Release 17.0.1as, analyze_system Release 17.0.1asCurrent process is 357, ptep 8FA86000, Noah_Davids.SysAdmin
Changing maximum tcp segment size (min_mss)from 556 to 1480ready 10:52:24
The parameter name, min_mss, implies that it is the MSS value but it is really the MSS value plus the TCP header length of 20 bytes. For the above example you would set the min_mss value to 556+20 or 576.
So how much of a boost can you get by increasing the MSS value? Because of the prevalence of VPN connections which add overhead bytes, I do not recommend setting the maximum value of 1480 (actual TCP MSS of 1460). I suggest something a little smaller, say 1220 (actual MSS 1200) bytes. Assuming you are sending 10 megabytes of data using 536 byte segments will require 18657 frames, total number of bytes – including overhead (8 byte Ethernet preamble + 14 byte Ethernet header + 20 byte IP header + 20 byte TCP header + 4 byte Ethernet trailer + an effective 12 bytes for the Ethernet inter-frame gap) will be 11,455,246 bytes. If you are using 1200 byte segments it will require 8334 frames with a total number of bytes 10,650,052 or a little over 9% smaller. However, keep in mind that the MSS value is an advertisement of the largest segment that will be accepted, there is no requirement that the sending host actually send segments that large. However, assuming that it does and that there are no other bottle necks you just got a 9% boot in throughput.
Can we do anything on the UDP side?
UDP does not have the concept of maximum segment size so there is no parameter to adjust it.