Summary
TCP/IP Illustrated, Volume 3 covers four major topics of great importance to anyone working TCP/IP. It contains the first thorough treatment of TCP for transactions, commonly known as T/TCP, an extension to TCP that makes client-server transactions faster and more efficient. Next, the book covers two popular applications of T/TCP, the very hot topic of HTTP (the Hypertext Transfer Protocol), the foundation for the World Wide Web, and NNTP (the Network News Transfer Protocol), the basis for the Usenet news system. Both of these topics have increased in significance as the Internet has exploded in size and usage. Finally, the book covers UNIX Domain Protocols, protocols that are used heavily in UNIX implementations.
Author Biography
We are deeply saddened to learn of the death of noted author W. Richard Stevens. His passing is obviously a tremendous loss for the technical community, but it is a personal one for us as well. Rich was both a gifted colleague and a valued friend who will be greatly missed. We extend our sympathies to his family.
Obituary from the Arizona Daily Star:
STEVENS, W. Richard, noted author of computer books died on September 1. He is best known for his "UNIX Network Programming" series (1990, 1998, 1999), "Advanced Programming in the UNIX Environment" (1992), and "TCP/IP Illustrated" series (1994, 1995, 1996). Richard was born in 1951 in Luanshya, Northern Rhodesia (now Zambia), where his father worked for the copper industry. The family moved to Salt Lake City, Hurley, New Mexico, Washington, DC and Phalaborwa, South Africa. Richard attended Fishburne Military School in Waynesboro, Virginia. He received a B.SC. in Aerospace Engineering from the University of Michigan in 1973, and an M.S. (1978) and Ph.D. (1982) in Systems Engineering from the University of Arizona. He moved to Tucson in 1975 and from then until 1982 he was employed at Kitt Peak National Observatory as a computer programmer. From 1982 until 1990 he was Vice President of Computing Services at Health Systems International in New Haven, CT, moving back to Tucson in 1990. Here he pursued his career as an author and consultant. He was also an avid pilot and a part-time flight instructor during the 1970's. He is survived by his loving wife of 20 years, Sally Hodges Stevens; three wonderful children, Bill, Ellen and David; sister, Claire Stevens of Las Vegas, NV; brother, Bob and wife Linda Stevens of Dallas, TX; nieces, Laura, Sarah, Collette, Christy; and nephew, Brad. He is predeceased by his parents, Royale J. Stevens (1915-1984); and Helen Patterson Stevens (1916-1997). Helen lived in Tucson from 1991-1997, and Royale lived here in the early 1930's attending Tucson High School while his father was treated for TB at the Desert Sanitorium (now TMC). The family asks that in lieu of flowers, donations be made in Richard's name to Habitat for Humanity, 2950 E. 22nd Street, Tucson, AZ 85713. A memorial service for Richard will be held at St. Phillip's in the Hills Episcopal Church on Tuesday, September 7th at 12:00 noon. Following the service there will be a reception in the Murphy Gallery of the Church. Please wear colorful clothing to the service; Richard loved colors.
W. Richard Stevens was an acknowledged UNIX and networking expert and the highly-respected author of several books. He was also a sought-after instructor and consultant.
0201634953AB04062001
Table of Contents
Preface |
|
xv | |
Part 1. TCP for Transactions |
|
1 | (158) |
|
|
3 | (26) |
|
|
3 | (1) |
|
|
3 | (6) |
|
|
9 | (8) |
|
|
17 | (3) |
|
|
20 | (1) |
|
|
21 | (1) |
|
|
22 | (2) |
|
|
24 | (2) |
|
|
26 | (2) |
|
|
28 | (1) |
|
|
29 | (10) |
|
|
29 | (1) |
|
New TCP Options for T/TCP |
|
|
30 | (3) |
|
T/TCP Implementation Variables |
|
|
33 | (1) |
|
|
34 | (2) |
|
|
36 | (2) |
|
|
38 | (1) |
|
|
39 | (14) |
|
|
39 | (1) |
|
|
40 | (2) |
|
|
42 | (1) |
|
Server Receives Old Duplicate SYN |
|
|
43 | (1) |
|
|
44 | (1) |
|
Request or Reply Exceeds MSS |
|
|
45 | (4) |
|
|
49 | (2) |
|
|
51 | (2) |
|
T/TCP Protocol (Continued) |
|
|
53 | (16) |
|
|
53 | (1) |
|
Client Port Numbers and TIME_WAIT State |
|
|
53 | (3) |
|
Purpose of the TIME_WAIT State |
|
|
56 | (3) |
|
TIME_WAIT State Truncation |
|
|
59 | (3) |
|
Avoiding the Three_Way Handshake with TAO |
|
|
62 | (6) |
|
|
68 | (1) |
|
T/TCP Implementation: Socket Layer |
|
|
69 | (4) |
|
|
69 | (1) |
|
|
70 | (1) |
|
|
70 | (2) |
|
|
72 | (1) |
|
T/TCP Implementation: Routing Table |
|
|
73 | (14) |
|
|
73 | (1) |
|
|
74 | (1) |
|
radix_node_head Structure |
|
|
75 | (1) |
|
|
75 | (1) |
|
|
76 | (1) |
|
|
76 | (1) |
|
|
77 | (1) |
|
|
78 | (1) |
|
|
78 | (1) |
|
|
79 | (3) |
|
|
82 | (3) |
|
|
85 | (2) |
|
T/TCP Implementation: Protocol Control Blocks |
|
|
87 | (4) |
|
|
87 | (1) |
|
|
88 | (1) |
|
|
89 | (1) |
|
|
90 | (1) |
|
T/TCP Implementation: TCP Overview |
|
|
91 | (6) |
|
|
91 | (1) |
|
|
91 | (1) |
|
|
92 | (1) |
|
|
93 | (1) |
|
|
94 | (1) |
|
|
94 | (1) |
|
|
95 | (2) |
|
T/TCP Implementation: TCP Output |
|
|
97 | (8) |
|
|
97 | (1) |
|
|
97 | (7) |
|
|
104 | (1) |
|
T/TCP Implementation: TCP Functions |
|
|
105 | (20) |
|
|
105 | (1) |
|
|
105 | (1) |
|
|
106 | (2) |
|
|
108 | (1) |
|
Retransmission Timeout Calculations |
|
|
108 | (4) |
|
|
112 | (1) |
|
|
113 | (1) |
|
|
114 | (7) |
|
|
121 | (1) |
|
|
122 | (2) |
|
|
124 | (1) |
|
T/TCP Implementation: TCP Input |
|
|
125 | (24) |
|
|
125 | (1) |
|
|
125 | (4) |
|
|
129 | (1) |
|
Initiation of Passive Open |
|
|
130 | (4) |
|
Initiation of Active Open |
|
|
134 | (7) |
|
PAWS: Protection Against Wrapped Sequence Numbers |
|
|
141 | (1) |
|
|
142 | (1) |
|
Completion of Passive Opens and Simultaneous Opens |
|
|
142 | (1) |
|
ACK Processing (Continued) |
|
|
143 | (2) |
|
|
145 | (2) |
|
|
147 | (2) |
|
T/TCP Implementation: TCP User Requests |
|
|
149 | (10) |
|
|
149 | (1) |
|
|
149 | (1) |
|
|
150 | (4) |
|
PRU_SEND and PRU_SEND_EOF Requests |
|
|
154 | (1) |
|
|
155 | (1) |
|
|
155 | (1) |
|
|
156 | (2) |
|
|
158 | (1) |
Part 2. Additional TCP Applications |
|
159 | (60) |
|
HTTP: Hypertext Transfer Protocol |
|
|
161 | (16) |
|
|
161 | (1) |
|
Introduction to HTTP and HTML |
|
|
162 | (3) |
|
|
165 | (5) |
|
|
170 | (2) |
|
|
172 | (1) |
|
|
173 | (2) |
|
|
175 | (2) |
|
Packets Found on an HTTP Server |
|
|
177 | (30) |
|
|
177 | (3) |
|
|
180 | (1) |
|
Client SYN Interarrival Time |
|
|
181 | (4) |
|
|
185 | (2) |
|
|
187 | (5) |
|
|
192 | (3) |
|
Client SYN Retransmissions |
|
|
195 | (1) |
|
|
196 | (1) |
|
Timing Out Persist Probes |
|
|
196 | (4) |
|
Simulation of T/TCP Routing Table Size |
|
|
200 | (2) |
|
|
202 | (1) |
|
TCP PCB Cache and Header Prediction |
|
|
203 | (2) |
|
|
205 | (2) |
|
NNTP: Network News Transfer Protocol |
|
|
207 | (12) |
|
|
207 | (2) |
|
|
209 | (3) |
|
|
212 | (2) |
|
A More Sophisticated News Client |
|
|
214 | (1) |
|
|
215 | (1) |
|
|
216 | (3) |
Part 3. The Unix Domain Protocols |
|
219 | (72) |
|
Unix Domain Protocols: Introduction |
|
|
221 | (6) |
|
|
221 | (1) |
|
|
222 | (1) |
|
|
223 | (1) |
|
|
224 | (1) |
|
|
225 | (2) |
|
Unix Domain Protocols: Implementations |
|
|
227 | (36) |
|
|
227 | (1) |
|
|
227 | (1) |
|
Unix domain and protosw Structures |
|
|
228 | (2) |
|
Unix Domain Socket Address Structures |
|
|
230 | (1) |
|
Unix Domain Protocol Control Blocks |
|
|
231 | (2) |
|
|
233 | (1) |
|
PRU_ATTACH Request and unp_attach Function |
|
|
233 | (3) |
|
PRU_DETACH Request and unp_detach Function |
|
|
236 | (1) |
|
PRU_BIND Request and unp_bind Function |
|
|
237 | (3) |
|
PRU_CONNECT Request and unp_connect Function |
|
|
240 | (5) |
|
PRU_CONNECT2 Request and unp_connect2 Function |
|
|
245 | (4) |
|
|
249 | (4) |
|
|
253 | (1) |
|
|
253 | (2) |
|
PRU_DISCONNECT Request and unp_disconnect Function |
|
|
255 | (2) |
|
PRU_SHUTDOWN Request and unp_shutdown Function |
|
|
257 | (1) |
|
PRU_ABORT Request and unp_drop Function |
|
|
258 | (1) |
|
|
259 | (2) |
|
|
261 | (2) |
|
Unix Domain Protocols: I/O and Descriptor Passing |
|
|
263 | (28) |
|
|
263 | (1) |
|
PRU_SEND and PRU_RCVD Requests |
|
|
263 | (6) |
|
|
269 | (5) |
|
|
274 | (2) |
|
|
276 | (1) |
|
|
277 | (1) |
|
|
278 | (1) |
|
|
278 | (2) |
|
|
280 | (8) |
|
|
288 | (1) |
|
|
288 | (1) |
|
|
289 | (2) |
Appendix A. Measuring Network Times |
|
291 | (12) |
|
A.1 RTT Measurements Using Ping |
|
|
292 | (2) |
|
A.2 Protocol Stack Measurements |
|
|
294 | (6) |
|
A.3 Latency and Bandwidth |
|
|
300 | (3) |
Appendix B. Coding Applications for T/TCP |
|
303 | (6) |
Bibliography |
|
309 | (6) |
Index |
|
315 | |
Excerpts
Introduction and Organization of the Book This book is a logical continuation of theTCP/IP Illustratedseries: Stevens 1994, which we refer to asVolume 1, and Wright and Stevens 1995, which we refer to asVolume 2.This book is divided into three parts,each covering a different topic:TCP for transactions, commonly called T/TCP. This is an extension to TCP designed to make client-server transactions faster, more efficient, and reliable. This is done by omitting TCP's three-way handshake at the beginning of a connection and shortening the TIME_WAIT state at the end of a connection. We'll see that T/TCP can match UDP's performance for a client-server transaction and that T/TCP provides reliability and adaptability, both major improvements over UDP.A transaction is defined to be a client request to a server, followed by the server's reply. (The term transaction does not mean a database transaction, with locking, two-phase commit, and backout.)TCP/IP applications, specifically HTTP (the Hypertext Transfer Protocol, the foundation of the World Wide Web) and NNTP (the Network News Transfer Protocol, the basis for the Usenet news system).The Unix domain protocols. These protocols are provided by all Unix TCP/IP implementations and on many non-Unix implementations. They provide a form of interprocess communication (IPC) and use the same sockets interface used with TCP/IP. When the client and server are on the same host, the Unix domain protocols are often twice as fast as TCP/IP.Part 1, the presentation of T/TCP, is in two pieces. Chapters 1-4 describe the protocol and provide numerous examples of how it works. This material is a major expansion of the brief presentation of T/TCP in Section 24.7 ofVolume 1. The second piece, Chapters 5-12, describes the actual implementation of T/TCP within the 4.4BSD-Lite networking code (i.e., the code presented inVolume 2). Since the first T/TCP implementation was not released until September 1994, about one year afterVolume 1was published and right asVolume 2was being completed, the detailed presentation of T/TCP, with examples and all the implementation details, had to wait for another volume in the series.Part 2, the HTTP and NNTP applications, are a continuation of the TCP/IP applications presented in Chapters 25-30 ofVolume 1. In the two years sinceVolume 1was published, the popularity of HTTP has grown enormously, as the Internet has exploded, and the use of NNTP has been growing about 75% per year for more than 10 years. HTTP is also a wonderful candidate for T/TCP, given its typical use of TCP: short connections with small amounts of data transferred, where the total time is often dominated by the connection setup and teardown. The heavy use of HTTP (and therefore TCP) on a busy Web server by thousands of different and varied clients also provides a unique opportunity to examine the actual packets at the server (Chapter 14) and look at many features of TCP/IP that were presented inVolumes 1 and 2.The Unix domain protocols in Part 3 were originally considered forVolume 2but omitted when its size reached 1200 pages. While it may seem odd to cover protocols other than TCP/IP in a series titledTCP/IP Illustrated, the Unix domain protocols were implemented almost 15 years ago in 4.2BSD alongside the first implementation of BSD TCP/IP. They are used heavily today in any Berkeley-derived kernel, but their use is typically "under the covers," and most users are unaware of their presence. Besides being the foundation for Unix pipes on a Berkeley-derived kernel, another heavy user is the X Window System, when the client and server are on the same host (i.e., on typical workstations). Unix domain sockets are also used to pass descriptors between processes, a powerful technique for interprocess communication. Since the sockets API (application program interface) used with the Unix domain prot