2.1 - Which sockets API should I target?

From the programmer’s standpoint, there are two major versions of Winsock, 2.0 and 1.1. These are the version numbers you pass to WSAStartup(), and your choice affects whether you #include <winsock2.h> and link against ws2_32.dll or you #include <winsock.h> and link against wsock32.dll. You could choose to declare your program as using some other minor version, but unless you know for a fact that you need to use something different, I recommend using only 1.1 or 2.0, for best compatibility.

The Winsock design is a series of additions to older technologies, not something wholly new. Winsock 2.0 is mostly a strict superset of Winsock 1.1, and Winsock 1.1 is mostly a strict superset of BSD Sockets. (In both cases, there are some things left behind or which simply change. I cover these things elsewhere in the FAQ.)

The older APIs are still quite useful. If your program doesn’t need anything added in later versions of the API, it’s perfectly fine to limit your program to only Winsock 1.1 or to portable BSD Sockets.

The Winsock 1.1 specific DLLs are still provided on the newest versions of Windows. Winsock 1.1 has some limitations compared to Winsock 2, but there are few types of programs that absolutely require Winsock 2 features.

Likewise, BSD Sockets, though a very old API now, still contains a large amount of functionality. Most network programs running today on Linux, Unix, and OS X still use nothing but the plain old BSD Sockets API. Viewed that way, even Winsock 1.1 looks modern, new, and powerful. If your program doesn’t need any of the Windows-specific extensions, or simply has to be portable to these other operating systems, feel free to stick to the BSD Sockets subset of the API. There are some details that prevent you from simply recompiling a BSD Sockets program on Windows, but these are generally easy to work around.

I’m not aware of any penalty for choosing an older API on a newer operating system. Nor for that matter am I aware of penalties for using an older API subset while building your program against Winsock 2. There is the decreased portability, of course, but since there are Winsock 2 options back to Windows 95, it’s not generally a practical concern.

The FAQ’s example programs are all set up to build as Winsock 2 code just to be modern. That said, you could easily make most of them build against Winsock 1.1 simply by changing the things mentioned at the top of this FAQ item.

2.2 - Are there any sample apps on the Net?

Yes. There are several listed on the Resources page, and the FAQ’s Examples section has several more. If you’re just getting started with Winsock, you may be especially interested in these samples.

2.3 - Do I need to initialize the WSAData structure before calling WSAStartup?

No, WSAStartup() fills this structure in for you.

2.4 - I’m getting link errors when compiling Winsock programs. What’s wrong?

You’re most likely not linking with the proper Winsock import library. This is ws2_32.lib if you’re targetting Winsock 2, or wsock32.lib for Winsock 1.1.

2.5 - If I write a Winsock program, will I be able to communicate with a Unix sockets program?

Absolutely! This common question is the result of confusing protocols with the APIs. Communicating programs need not have been created with the same APIs, as long as they are using the same transport and network protocols.

Before dealing with cross-platform networking, please read the FAQ article How to Use TCP Effectively. It covers several issues that bite cross-platform programs, like structure padding and data representation.

2.6 - Can I use Winsock with { My Favorite Language }?

Maybe, but Winsock is usually only used directly in low-level languages like C and its close relatives. There are several reasons for this.

Reason 1: Some languages simply lack the language features to call the Winsock API directly.

A programming language needs the following features to access Winsock:

1. Pointers. (The ability to access a specific piece of memory by its address.)

2. Bitwise operators. (The ability to change specific bits in a byte.)

3. Structures or records. (The ability to define a block of memory that is an aggregate of simple data elements, such as two characters followed by a 16-bit integer. This feature must also allow some measure of control as to how the data is laid out in memory.)

Most high-level languages lack one or more of these features. Your language might have some kind of extension mechanism that lets you write code in C or a similar language to provide a bridge, but that is usually not necessary.

Reason 2: Most modern languages commonly used on Windows provide network support as part of the language or its runtime. This is typical of cross-platform languages like Java, Perl, Python, Tcl, Ruby, Lua... Even more specialized languages, like R, Mathematica, and Erlang usually include network support. Unless you are using a language without network support built in or need to dig into some Winsock-specific mechanism not exposed by the language’s network support, you won’t have cause to use Winsock directly. Using the language’s own network API usually results in smaller, more portable code.

Reason 3: Older languages often didn’t have networking as part of the language, but had some extension mechanism whereby networking was later added. A good example is old versions of Visual Basic, and the now-moribund market for ActiveX networking controls. Other than this external component difference, the argument follows that for Reason 2.

For these reasons and others, this FAQ is biased towards C and C++. It probably also applies just as well to other close C family members like Objective C and D, but I don’t use those, so I haven’t tried to ensure code compatibility.

If your language allows direct access to the Winsock API, you may be able to translate the C++ code in the FAQ into equivalent code in your chosen language. However, you’ll probably do better to try and find sample code in your chosen language elsewhere on the net. You may be able to find something that does exactly what you need, and if not, at least you will have a basis in your language’s network APIs which will make it easier to translate the code here into your language.

2.7 - Are there any tools available for debugging Winsock programs?

The two main categories of debugging tools are sniffers and shims.

Sniffers

I’ve moved all discussion of what sniffers are, their limitations, and workarounds for those limitations into a separate article, “The Straight Dope on Packet Sniffers.”

The main advantage of a sniffer is that it literally sees every bit of data in a network conversation, way down below the Winsock level to the network hardware’s protocol layer. (e.g. Ethernet) Another advantage is that a good sniffer is quite powerful and configurable. For example, some allow you to write “protocol plugins” that will decode any protocol, such as a custom protocol that you’ve developed. You can also build complex expressions to filter out just the small bit of traffic you need to see to debug a problem, so you don’t have to wade through the torrent of variegated data on a modern LAN.

By far, the most popular packet sniffer today is the open source and cross-platform Wireshark suite. (Previously called Ethereal.) In the same way that Firefox slowly, unevenly, yet inevitably steamrolled its commercial competition, Wireshark has pushed out all but the very best commercial offerings. It is not perfect, but it is more than good enough for day-to-day packet sniffing; I use it almost exclusively.

The main commercial sniffers left are WildPackets’ OmniPeek, Network Instruments’ Observer, and NetScout’s Sniffer. (An ancestor of the latter is the source of the now-kleenexed term “sniffer.”) These all cost north of US$ 1,000 and run into deep the thousands for enhanced versions and optional add-ons. I haven’t used any of them in many years, so I cannot advise about the conditions where they are worth the cost. (They must be, else they would not have survived this long in the face of a good-enough free alternative.)

The only other sniffer I use is tcpdump. Its main virtue is that it’s installed by default on almost every Linux, Unix and OS X box out there. You can usually find such a box somewhere nearby when you need to debug something. I almost never use it to analyze captured packets; its display format is difficult to read and often doesn’t tell you enough besides. Instead, I use its -w option to write the captured packets to a file, which I then compress, transfer back to my PC, then load up in Wireshark for analysis. I can do this even when not on-site with the problem by dialing or sshing into a *ix box on site. Beware that you might need to give the -s option to get it to capture more than just the packet headers.

You can use Wireshark’s tshark command-line companion for this instead, if available. Its output format is more readable and it captures whole packets by default.

Shims

Another major category of debugging tools are shims. (Also called spies.) A shim uses any of several different methods to monitor a single application’s use of the Winsock APIs. Shims present a higher-level view of your program’s network interactions than sniffers.

The difference in perspective can make it easier to map what it displays onto what you know your program is trying to do. Captured packets are sometimes too far removed from the problem source to be immediately helpful.

(The reverse can be true, too: knowing the sequence of API calls doesn’t tell you every detail about the data that is actually going out on the wire. Sniffers tell you that.)

Shims can sometimes work where sniffers will not. Some types of point-to-point links (e.g. PPP over a modem) are sniff-resistant due to their nature, but if they’re exposed to Winsock, a shim can see them. Some types of shims work in such a way that they don’t require administrator privileges on the local machine, unlike most sniffers.

The main shims available are Systems Software Technology’s TracePlus/Winsock (US $160) and WinTECH’s SocktSpy (US $60).

Miscellaneous Tools

There are various other network debugging tools out there useful to programmers.

Every network programmer needs a port scanner, with which you can probe a machine to learn what ports are open, what ports are closed or blocked by a firewall, even what OS a remote machine is running based on how it responds to the scanning packets. One nice GUI tool of that sort is Atelier Web’s Security Port Scanner, a.k.a. AWSPS. (US $34 to 120, depending on options) A more powerful but less pretty open source alternative is Nmap.

Another useful class of tool is something that tells you which programs on a given machine are listening on which ports. Merely knowing that something is listening on TCP port 8472 isn’t as useful as knowing what is listening on that port. AWSPS does this, as well. One free but closed-source tool to do this is SysInternals’ TCPView. Another is Vision, from security firm Foundstone. In Windows XP and up, you can get a primitive form of this information at the command line by saying netstat -ba. The -b option is Windows-specific; to get the same information on Linux, BSD, and OS X systems, look into the awesome lsof command. (The example question posed above would be answered with lsof -i:8472.)

General Advice

No one tool will do everything you need. You might use a port scanner to check for an open port, start a sniffer capturing packets on that port, point your program at it, capture the data, then go back and run it through a shim to find out why the API calls you’re making are giving you a some weird network packets the sniffer captured.

I suggest that you start with Wireshark, Nmap and TCPView, then explore the commercial alternatives’ demos to see if they add any features you just gotta have. I find less use for shims myself, but that may be a result of long experience; I rarely find myself at a loss to explain why a particular packet capture was produced by a given program.

You may also find the FAQ article Debugging TCP/IP useful for some less-automated methods of debugging a TCP program.

Methods That Don’t Work

There are a couple of debugging tools that are supposed to work that don’t, or are too flaky to deal with. The first is the SO_DEBUG socket option. It simply doesn’t work on Microsoft stacks. The other is the Winsock DLL debugging plugin dt_dll.dll; this method is flaky. Bob Quinn wrote an article about this, but unfortunately the site that held it was bought by another company that hasn’t yet made that article available again.

2.8 - How do I get a readable error message from a Winsock error number?

The problem with this question is that it assumes that there is a “good” canned error message for every situation. The reality is that many times, you need to know the program’s context before you can turn an error value into a meaningful error message. For example, WSAEFAULT can mean any of the following, depending on context:

• bad pointer passed
• passed buffer too small
• that version of the API is not supported

Since the Winsock docs tell you the most likely error values that each function will return, you should use this information to construct intelligent error handlers.

Still, sometimes an API call returns something unexpected, so a cryptic error message is better than none at all. For such “default” case handling the FormatMessage() API on all modern versions of Windows (except NT 4) will return canned messages when given Winsock error numbers.

I must stress that you’re better off spending your time constructing meaningful error messages based on program context than chasing something that doesn’t work very well even in the best case. Use canned messages only when your program has no better idea what to do about an error.

2.9 - Winsock keeps returning the error WSAEWOULDBLOCK. What’s wrong with my program?

Not a thing. WSAEWOULDBLOCK is a perfectly normal occurrence in programs using non-blocking and asynchronous sockets. It’s Winsock’s way of telling your program “I can’t do that right now, because I would have to block to do so.”

The next question is, how do you know when it’s safe to try again? In the case of asynchronous sockets, Winsock will send you an FD_WRITE message after a failed send() call when it is safe to write; it will send you an FD_READ message after a recv() call when more data arrives on that socket. Similarly, in a non-blocking sockets program that uses select(), the writefds will be set when it’s okay to write, and the readfds will be set if there is data to read.

Don’t count on Winsock to handle blocking the way you think it ought to. Whenever you use some form of nonblocking sockets, you have to be prepared for WSAEWOULDBLOCK at any time. It’s simply a matter of defensive programming, just like checking for null pointers.

2.10 - How can I test my Winsock application without setting up a network?

There is a special address called the loopback or localhost address, 127.0.0.1. This lets two programs running on a single machine talk to each other. The server usually listens for connections on all available interfaces, and the client connects to the localhost address. (See the Example Programs section for basic client and server program code.)

If you have an Internet or LAN connection on your development machine, you’re already set up for this.

For machines with no network at all, you have to set up a “dummy” network. On modern versions of Windows, you can go into the Network control panel, tell it to add an interface, and select the Microsoft Loopback Device. You don’t need any other networking support for this to work.

Be warned: behavior through the loopback interface will differ from that on a “real” network, if only because conditions are much simpler within a single machine than over a LAN or WAN. You should try to test your application on a real network, even if you do primary development on a single machine.

2.11 - What’s the proper way to close a TCP socket?

TCP is a bidirectional protocol. Think about it like a conversation between two people. It’s rude for one person to simply stop talking and walk away. Proper social protocol requires that the person who is done talking wait and listen to what the other person has to say before walking away. One side starts winding down the conversation — “Well, it’s been nice, but I have to go.” — then politely waits for the other person to finish and agree to finish. (“Yes, okay, I’ll see you later.”)

TCP connections with Winsock are designed to work the same way:

1. One side finishes sending data. It tells this to the stack by calling shutdown() with the how parameter set to SD_SEND.

2. Both sides continue to call recv(). The side that closed its sending half of the connection must because the remote peer probably isn’t done sending yet. The peer that is still sending must continue calling recv() because it will get 0 when the remote peer calls shutdown() with SD_SEND. If you are using asynchronous sockets or event objects, you will get an FD_CLOSE when this happens, too. In programs that use select(), you simply get a signaled socket in the readfds, which tells you to call recv(), whereupon you get the 0 return value. Beware that both sides still have to be prepared to receive a -1 return from recv(), indicating an error.

3. Both sides call closesocket() only when they are convinced the conversation is over, either because they have stopped sending and they got 0 from recv(), or they got a connection error.

Not following this graceful shutdown protocol can cause data loss.

Nonblocking or asynchronous sockets complicate this. You can either build “finish sending/receiving” logic into your normal I/O loop, or you can temporarily put the socket in blocking mode and do the last bits of I/O that way. The proper choice depends on your program’s architecture and requirements.

2.12 - Is it possible to close the connection “abnormally”?

Sure, but it’s an evil thing to do. :)

The simplest way is to call shutdown() with the how parameter set to SD_BOTH (“both directions”), optionally followed by a closesocket() call. Another method is to set the SO_LINGER flag to 0 with the setsockopt() call before you call closesocket(). This causes a TCP connection reset. (That is, a TCP packet goes out with the RST flag set, forcing the remote stack to drop the connection immediately.)

“Slamming the connection shut” is only justifiable in a very small number of cases. You must have fairly deep knowledge of the way TCP works before you can properly decide to use this technique. Generally, the perceived need to slam the connection shut comes from a broken program, either yours or the remote peer. I recommend that you try to fix the broken program so you don’t have to resort to such a questionable technique.

2.13 - How do I detect when my TCP connection is closed?

All of the I/O strategies discussed in the I/O strategies article have some way of indicating that the connection is closed.

First, keep in mind that TCP is a full-duplex network protocol. That means that you can close the connection half-way and still send data on the other half. An example of this is the old HTTP 1.0 web protocol. The browser sends a short request to the web server, then closes its half of the connection. The web server then sends back the requested data on the other half of the connection, and closes its sending side, which terminates the TCP session. (HTTP 1.1 is more complex than this, but at the end, the same basic thing happens.)

Normal TCP programs only close the sending half, which the remote peer perceives as the receiving half. So, what you normally want to detect is whether the remote peer closed its sending half, meaning you won’t be receiving data from them any more.

With asynchronous sockets, Winsock sends you an FD_CLOSE message when the connection drops. Event objects are similar: the system signals the event object with an FD_CLOSE notification.

With blocking and non-blocking sockets, you probably have a loop that calls recv() on that socket. recv() returns 0 when the remote peer closes the connection. As you would expect, if you are using select(), the SOCKET descriptor in the read_fds parameter gets set when the connection drops. As normal, you’ll call recv() and see the 0 return value.

As you might have guessed from the discussion above, it is also possible to close the receiving half of the connection. If the remote peer then tries to send you data, the stack will drop that data on the floor and send a TCP RST to the remote peer.

See below for information on handling abnormal disconnects.

2.14 - How do I detect an abnormal network disconnect?

The previous question deals with detecting when a protocol connection is dropped normally, but what if you want to detect other problems, like unplugged network cables or crashed workstations? In these cases, the failure prevents notifying the remote peer that something is wrong. My feeling is that this is usually a feature, because the broken component might get fixed before anyone notices, so why demand that the connection be reestablished?

If you have a situation where you must be able to detect all network failures, you have two options:

The first option is to give the protocol a command/response structure: one host sends a command and expects a prompt response from the other host when the command is received or acted upon. If the response does not arrive, the connection is assumed to be dead, or at least faulty.

The second option is to add an “echo” function to your protocol, where one host (usually the client) is expected to periodically send out an “are you still there?” packet to the other host, which it must promptly acknowledge. If the echo-sending host doesn’t receive its response or the receiving host fails to see an echo request for a certain period of time, the program can assume that the connection is bad or the remote host has gone down.

If you choose the “echo” alternative, avoid the temptation to use the ICMP “ping” facility for this. If you did it this way, you would have to send pings from both sides, because Microsoft stacks won’t let you see the other side’s echo requests, only responses to your own echo requests. Another problem with ping is that it’s outside your protocol, so it won’t detect a failed TCP connection if the hardware connection remains viable. A final problem with the ping technique is that ICMP is an unreliable protocol: does it make a whole lot of sense to use an unreliable protocol to add an assurance of reliability to another protocol?

If you can restrict your program to Windows 2000 and its direct descendants, one option you might consider is TCP keepalives. This is a way to tell the stack to send a packet out over the connection at specific intervals whether there’s real data to send or not. If the remote host is up, it will send back a similar reply packet. If the TCP connection is no longer valid (e.g. the remote host has rebooted since the last keepalive), the remote host will send back a reset packet, killing the local host’s connection. If the remote host is down, the local host’s TCP stack will time out waiting for the reply and kill the connection. The main problem with keepalives is that these packets are entirely wasteful: they carry no useful data. At least with the command/response option, you have the possibility that every packet carries meaning.

Note that different types of networks handle physical disconnection differently. Ethernet, for example, establishes no link-level connection, so if you unplug the network cable, a remote host can’t tell that its peer is physically unable to communicate. By contrast, a dropped PPP link causes a detectable failure at the link layer, which propagates up to the Winsock layer for your program to detect.

2.15 - How can I change the timeout for a Winsock function?

Some of the blocking Winsock functions (e.g. connect()) have a timeout embedded into them. The theory behind this is that only the stack has all the information necessary to set a proper timeout. Yet, some people find that the value the stack uses is too long for their application; it can be a minute or longer.

You can adjust the send() and recv() timeouts with the SO_SNDTIMEO and SO_RCVTIMEO setsockopt() options. .

For other Winsock functions, the best solution is to avoid blocking sockets altogether. All of the non-blocking socket methods provide ways for you to build custom timeouts:

• Non-blocking sockets with select() – The fifth parameter to the select() function is a timeout value.

• Asynchronous sockets – Use the Windows API SetTimer().

• Event objects – WSAWaitForMultipleEvents() has a timeout parameter.

• Waitable Timers – Call CreateWaitableTimers() to make a waitable timer, which you can then pass to a function like WSAEventSelect() along with your sockets: if none of the sockets is signalled before the timer goes off, the blocking function will return anyway.

Note that with asynchronous and non-blocking sockets, you may be able to avoid handling timeouts altogether. Your program continues working even while Winsock is busy. So, you can leave it up to the user to cancel an operation that’s taking too long, or just let Winsock’s natural timeout expire rather than taking over this functionality in your code.

2.16 - What is peeking (MSG_PEEK), and why is it bad?

Peeking is looking ahead in the TCP data stream: when you use the MSG_PEEK flag with recv(), it returns bytes from the stack’s buffer without removing these from the buffer. (You can also do a form of peeking with the ioctlsocket() option FIONREAD.)

Peeking is essentially never necessary: you can always read data into your own buffers and process it there. This is good, because peeking often causes problems. Indeed, it’s so problematic it’s earned a place on the Lame List and in Microsoft’s Knowledge Base: see article KB192599 for specific info on the problems peeking causes with their Winsock stack.

2.17 - What is out-of-band data (MSG_OOB), and why is it bad?

Out-of-band (OOB) data is like a second data channel. The intent is to use the regular TCP data stream for most data and the OOB stream for “emergency” messages. The telnet protocol uses this for “interrupt” keystrokes like Ctrl-C, so that they don’t have to wait on the remote peer to handle regular TCP data before the interrupt occurs. You can send OOB data by passing the MSG_OOB flag to send() and receive it by passing MSG_OOB to recv(). You can also get OOB data by setting the SO_OOBINLINE flag with setsockopt().

OOB data is a useful concept, but unfortunately there are two conflicing interpretations of how OOB data should be handled at the stack level: the original description of OOB in the TCP protocol specification (RFC 793) was superceded by the “host requirements” spec (RFC 1122), but there are still many machines with RFC 793 OOB implementations. Section 3.5.2 in the Winsock 2.2.2 spec provides the gory details on why RFC 793 vs. RFC 1122 is a problem.

OOB also isn’t a fully functional second data channel: it’s rather limited, even between two machines that agree on what OOB means.

So, never use OOB except when implementing legacy protocols like telnet which demand it. You can get reliable OOB-like behavior by simply using two data connections: one for normal data, and the second for emergency data.

2.18 - If MSG_PEEK and MSG_OOB are bad, what do I pass for send() and recv()’s flags parameter?

It’s perfectly valid to pass 0 for send() and recv()’s flags parameter.