C and C++ socket programming quick reference from networking basics to robust concurrent servers.
Socket Programming Cheatsheet
Section titled “Socket Programming Cheatsheet”Sockets are operating-system handles used to exchange bytes between processes. Most network programs follow the same shape:
Server: socket -> bind -> listen -> accept -> recv/send -> closeClient: socket -> connect ------------> send/recv -> closeUDP: socket -> bind (server) -> recvfrom/sendto -> closeThis page uses the POSIX/BSD sockets API on Linux and Unix-like systems. The same ideas apply to Windows Winsock; portability notes are included near the end.
1. Core Concepts
Section titled “1. Core Concepts”Protocol Layers
Section titled “Protocol Layers”| Layer | Examples | What Your Program Sees |
|---|---|---|
| Application | HTTP, DNS, your protocol | Messages and rules you define |
| Transport | TCP, UDP | Ports and byte delivery behavior |
| Network | IPv4, IPv6 | Source and destination addresses |
| Link | Ethernet, Wi-Fi, loopback | Usually handled by the OS |
Socket Vocabulary
Section titled “Socket Vocabulary”| Term | Meaning |
|---|---|
| IP address | Identifies a network interface, such as 127.0.0.1 or ::1 |
| Port | Identifies an application endpoint, from 0 to 65535 |
| Endpoint | Address plus port, such as 127.0.0.1:8080 |
| Listening socket | Server socket waiting for new TCP connections |
| Connected socket | One TCP conversation between two endpoints |
| Loopback | Local host only: 127.0.0.1 for IPv4, ::1 for IPv6 |
| Wildcard bind | Accept traffic for all local interfaces: 0.0.0.0 or :: |
| Backlog | Queue size hint for pending TCP connections |
File descriptor (fd) | Integer handle returned by POSIX socket calls |
TCP vs UDP
Section titled “TCP vs UDP”| Feature | TCP (SOCK_STREAM) | UDP (SOCK_DGRAM) |
|---|---|---|
| Connection | Required | No handshake required |
| Delivery | Reliable, ordered byte stream | Best-effort datagrams |
| Boundaries | Not preserved | One send corresponds to one datagram |
| Flow/congestion control | Built in | Application must cope |
| Typical use | HTTP, SSH, database clients | DNS, telemetry, games, discovery |
| Key trap | recv() may return only part of a message | Packets may be dropped, repeated, or reordered |
2. Headers, Types, and Compile Commands
Section titled “2. Headers, Types, and Compile Commands”Common POSIX Headers
Section titled “Common POSIX Headers”#include <arpa/inet.h> // inet_pton, inet_ntop, htons, ntohs#include <errno.h> // errno#include <fcntl.h> // fcntl, O_NONBLOCK#include <netdb.h> // getaddrinfo, freeaddrinfo, gai_strerror#include <netinet/in.h> // sockaddr_in, sockaddr_in6#include <poll.h> // poll#include <signal.h> // signal, SIGPIPE#include <stdint.h> // uint16_t, uint32_t#include <stdio.h> // perror, fprintf#include <stdlib.h> // exit#include <string.h> // memset, strlen#include <sys/socket.h> // socket, bind, listen, accept, connect#include <sys/time.h> // timeval#include <sys/types.h> // POSIX types#include <unistd.h> // closeEssential Types
Section titled “Essential Types”int fd; // Socket descriptorssize_t count; // Return type for send/recvsocklen_t addr_len; // Address structure lengthstruct sockaddr_storage addr; // Large enough for IPv4 or IPv6Compile
Section titled “Compile”cc -std=c17 -Wall -Wextra -Wpedantic -O2 server.c -o serverc++ -std=c++20 -Wall -Wextra -Wpedantic -O2 client.cpp -o clientOn Unix-like systems, sockets are normally part of the standard C library. Windows programs link with Ws2_32.lib.
3. Address Structures and Byte Order
Section titled “3. Address Structures and Byte Order”Integers sent on a network use network byte order (big-endian). Convert port numbers and binary integer fields at API boundaries.
uint16_t network_port = htons(8080); // Host to network, 16-bituint16_t host_port = ntohs(network_port);uint32_t network_value = htonl(value); // Host to network, 32-bituint32_t host_value = ntohl(network_value);struct sockaddr_in addr = {0};addr.sin_family = AF_INET;addr.sin_port = htons(8080);
// Bind only to local machine:inet_pton(AF_INET, "127.0.0.1", &addr.sin_addr);
// Or bind on every IPv4 interface:addr.sin_addr.s_addr = htonl(INADDR_ANY);struct sockaddr_in6 addr6 = {0};addr6.sin6_family = AF_INET6;addr6.sin6_port = htons(8080);addr6.sin6_addr = in6addr_any; // All local IPv6 interfacesPrint a Client Address Safely
Section titled “Print a Client Address Safely”void print_peer(const struct sockaddr *sa) { char ip[INET6_ADDRSTRLEN]; unsigned short port;
if (sa->sa_family == AF_INET) { const struct sockaddr_in *v4 = (const struct sockaddr_in *)sa; inet_ntop(AF_INET, &v4->sin_addr, ip, sizeof ip); port = ntohs(v4->sin_port); } else { const struct sockaddr_in6 *v6 = (const struct sockaddr_in6 *)sa; inet_ntop(AF_INET6, &v6->sin6_addr, ip, sizeof ip); port = ntohs(v6->sin6_port); } printf("peer=%s port=%hu\n", ip, port);}4. The Essential Socket Calls
Section titled “4. The Essential Socket Calls”| Call | Purpose | Typical Failure Checks |
|---|---|---|
socket(domain, type, protocol) | Create a socket | Returns -1 |
setsockopt(fd, ...) | Configure options | Returns -1 |
bind(fd, address, length) | Choose local address/port | Address in use, permission denied |
listen(fd, backlog) | Mark TCP socket as passive | Returns -1 |
accept(fd, ...) | Produce a new TCP client socket | Retry on EINTR |
connect(fd, ...) | Start TCP connection | Connection refused/timeout |
send(fd, data, len, flags) | Send connected bytes | Partial write or -1 |
recv(fd, data, len, flags) | Receive connected bytes | 0 means peer closed cleanly |
sendto(...) / recvfrom(...) | UDP send/receive | Datagrams may be truncated/lost |
shutdown(fd, how) | Close one or both directions | Useful for protocol endings |
close(fd) | Release descriptor | Always do this when finished |
Always check return values. Socket I/O is not guaranteed to complete an entire logical request in one call.
5. Minimal TCP Echo Server in C
Section titled “5. Minimal TCP Echo Server in C”This single-client server listens on 127.0.0.1:8080, echoes received bytes, and demonstrates the basic lifecycle.
#include <arpa/inet.h>#include <errno.h>#include <stdio.h>#include <stdlib.h>#include <string.h>#include <sys/socket.h>#include <unistd.h>
static void die(const char *message) { perror(message); exit(EXIT_FAILURE);}
static int send_all(int fd, const void *buffer, size_t length) { const char *p = buffer; size_t sent = 0;
while (sent < length) { ssize_t n = send(fd, p + sent, length - sent, 0); if (n > 0) { sent += (size_t)n; } else if (n < 0 && errno == EINTR) { continue; } else { return -1; } } return 0;}
int main(void) { int server_fd = socket(AF_INET, SOCK_STREAM, 0); if (server_fd < 0) die("socket");
int yes = 1; if (setsockopt(server_fd, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof yes) < 0) die("setsockopt");
struct sockaddr_in server = {0}; server.sin_family = AF_INET; server.sin_port = htons(8080); if (inet_pton(AF_INET, "127.0.0.1", &server.sin_addr) != 1) { fputs("Invalid listen address\n", stderr); close(server_fd); return EXIT_FAILURE; }
if (bind(server_fd, (struct sockaddr *)&server, sizeof server) < 0) die("bind"); if (listen(server_fd, 16) < 0) die("listen");
puts("Listening on 127.0.0.1:8080");
struct sockaddr_in peer; socklen_t peer_len = sizeof peer; int client_fd = accept(server_fd, (struct sockaddr *)&peer, &peer_len); if (client_fd < 0) die("accept");
char buffer[4096]; for (;;) { ssize_t n = recv(client_fd, buffer, sizeof buffer, 0); if (n > 0) { if (send_all(client_fd, buffer, (size_t)n) < 0) die("send"); } else if (n == 0) { puts("Client disconnected"); break; } else if (errno != EINTR) { die("recv"); } }
close(client_fd); close(server_fd); return 0;}cc -std=c17 -Wall -Wextra -O2 tcp_server.c -o tcp_server./tcp_serverFrom another terminal:
nc 127.0.0.1 80806. TCP Client with DNS and IPv4/IPv6 Support
Section titled “6. TCP Client with DNS and IPv4/IPv6 Support”Prefer getaddrinfo() over hard-coding address structures. It handles hostnames, IPv4, IPv6, and service/port conversion.
#include <errno.h>#include <netdb.h>#include <stdio.h>#include <stdlib.h>#include <string.h>#include <sys/socket.h>#include <unistd.h>
static int send_all(int fd, const void *buffer, size_t length) { const char *p = buffer; size_t sent = 0; while (sent < length) { ssize_t n = send(fd, p + sent, length - sent, 0); if (n > 0) sent += (size_t)n; else if (n < 0 && errno == EINTR) continue; else return -1; } return 0;}
int main(int argc, char **argv) { const char *host = argc > 1 ? argv[1] : "127.0.0.1"; const char *port = argc > 2 ? argv[2] : "8080"; const char message[] = "hello from client\n";
struct addrinfo hints = {0}; hints.ai_family = AF_UNSPEC; // Try IPv4 or IPv6 hints.ai_socktype = SOCK_STREAM;
struct addrinfo *results; int rc = getaddrinfo(host, port, &hints, &results); if (rc != 0) { fprintf(stderr, "getaddrinfo: %s\n", gai_strerror(rc)); return EXIT_FAILURE; }
int fd = -1; for (struct addrinfo *p = results; p != NULL; p = p->ai_next) { fd = socket(p->ai_family, p->ai_socktype, p->ai_protocol); if (fd < 0) continue; if (connect(fd, p->ai_addr, p->ai_addrlen) == 0) break; close(fd); fd = -1; } freeaddrinfo(results);
if (fd < 0) { fputs("Unable to connect\n", stderr); return EXIT_FAILURE; }
if (send_all(fd, message, sizeof message - 1) < 0) { perror("send"); close(fd); return EXIT_FAILURE; }
char buffer[4096]; ssize_t n = recv(fd, buffer, sizeof buffer - 1, 0); if (n > 0) { buffer[n] = '\0'; printf("reply: %s", buffer); } else if (n < 0) { perror("recv"); }
close(fd); return n < 0 ? EXIT_FAILURE : EXIT_SUCCESS;}cc -std=c17 -Wall -Wextra -O2 tcp_client.c -o tcp_client./tcp_client localhost 80807. TCP Is a Byte Stream: Define Framing
Section titled “7. TCP Is a Byte Stream: Define Framing”TCP delivers ordered bytes, not messages. If a sender performs two send() calls, a receiver may observe one combined recv(), several partial receives, or anything in between.
Common Framing Strategies
Section titled “Common Framing Strategies”| Strategy | Example | Good For | Caution |
|---|---|---|---|
| Delimiter | hello\n | Line commands, chat | Escape or reject delimiter in payload |
| Fixed size | Exactly 64 bytes | Simple binary records | Wastes space for variable content |
| Length prefix | uint32 length then payload | Binary/protobuf/JSON messages | Validate maximum length |
| Connection close | Read until EOF | Simple one-response protocols | Prevents reuse of connection |
Receive Exactly N Bytes
Section titled “Receive Exactly N Bytes”int recv_exact(int fd, void *buffer, size_t length) { char *p = buffer; size_t received = 0;
while (received < length) { ssize_t n = recv(fd, p + received, length - received, 0); if (n > 0) { received += (size_t)n; } else if (n == 0) { return 0; // Peer closed before a complete message arrived } else if (errno == EINTR) { continue; } else { return -1; } } return 1;}Length-Prefixed Message Pattern
Section titled “Length-Prefixed Message Pattern”#define MAX_MESSAGE (1024u * 1024u)
uint32_t network_length;int status = recv_exact(fd, &network_length, sizeof network_length);if (status <= 0) { /* Disconnect or error. */}
uint32_t length = ntohl(network_length);if (length > MAX_MESSAGE) { /* Reject oversized input before allocating or reading it. */}
char *payload = malloc((size_t)length + 1);if (payload == NULL) { /* Handle allocation failure. */}if (recv_exact(fd, payload, length) != 1) { /* Handle truncated input. */}payload[length] = '\0';Never deserialize or allocate based on an untrusted length without applying a limit.
8. UDP Datagram Server and Client
Section titled “8. UDP Datagram Server and Client”UDP is useful when messages are small and an application can tolerate loss or implement its own reliability rules.
UDP Echo Server
Section titled “UDP Echo Server”#include <arpa/inet.h>#include <stdio.h>#include <stdlib.h>#include <sys/socket.h>#include <unistd.h>
int main(void) { int fd = socket(AF_INET, SOCK_DGRAM, 0); if (fd < 0) { perror("socket"); return EXIT_FAILURE; }
struct sockaddr_in server = {0}; server.sin_family = AF_INET; server.sin_port = htons(8080); server.sin_addr.s_addr = htonl(INADDR_ANY);
if (bind(fd, (struct sockaddr *)&server, sizeof server) < 0) { perror("bind"); close(fd); return EXIT_FAILURE; }
for (;;) { char buffer[2048]; struct sockaddr_storage peer; socklen_t peer_len = sizeof peer; ssize_t n = recvfrom(fd, buffer, sizeof buffer, 0, (struct sockaddr *)&peer, &peer_len); if (n < 0) { perror("recvfrom"); continue; } if (sendto(fd, buffer, (size_t)n, 0, (struct sockaddr *)&peer, peer_len) < 0) { perror("sendto"); } }}UDP Client
Section titled “UDP Client”#include <arpa/inet.h>#include <stdio.h>#include <stdlib.h>#include <string.h>#include <sys/socket.h>#include <unistd.h>
int main(void) { int fd = socket(AF_INET, SOCK_DGRAM, 0); if (fd < 0) { perror("socket"); return EXIT_FAILURE; }
struct sockaddr_in server = {0}; server.sin_family = AF_INET; server.sin_port = htons(8080); if (inet_pton(AF_INET, "127.0.0.1", &server.sin_addr) != 1) { fputs("Invalid destination address\n", stderr); close(fd); return EXIT_FAILURE; }
const char request[] = "ping"; if (sendto(fd, request, sizeof request - 1, 0, (struct sockaddr *)&server, sizeof server) < 0) { perror("sendto"); close(fd); return EXIT_FAILURE; }
char reply[2048]; ssize_t n = recvfrom(fd, reply, sizeof reply - 1, 0, NULL, NULL); if (n < 0) { perror("recvfrom"); close(fd); return EXIT_FAILURE; } reply[n] = '\0'; printf("reply: %s\n", reply); close(fd); return EXIT_SUCCESS;}UDP Rules to Remember
Section titled “UDP Rules to Remember”- A UDP datagram has a maximum size, but large datagrams may be fragmented or dropped. Keep internet-facing application datagrams small when practical.
recvfrom()gives the sender address; validate it when replies should only come from a known peer.- If the receive buffer is smaller than the datagram, extra bytes are discarded.
- Add request IDs, retries, timeout handling, duplication checks, and rate limiting when your protocol needs them.
- Calling
connect()on a UDP socket does not create a TCP-style connection; it sets a default peer and filters unrelated inbound datagrams.
9. Errors, Interruptions, and Clean Shutdown
Section titled “9. Errors, Interruptions, and Clean Shutdown”Return Values
Section titled “Return Values”| Operation | Result | Meaning |
|---|---|---|
recv() | > 0 | Number of bytes received |
recv() | 0 | TCP peer performed an orderly shutdown |
recv() | -1, errno == EINTR | Signal interrupted call; usually retry |
recv() | -1, errno == EAGAIN or EWOULDBLOCK | Nonblocking socket has no data now |
send() | > 0 but < requested | Only part of the bytes were accepted |
send() | -1, errno == EPIPE | Peer has closed; may also trigger SIGPIPE |
Avoid Termination on SIGPIPE
Section titled “Avoid Termination on SIGPIPE”Writing to a closed TCP connection may terminate a POSIX process unless handled.
signal(SIGPIPE, SIG_IGN); // Process-wide optionOn systems that support it, passing MSG_NOSIGNAL to send() limits the behavior to that call:
send(fd, buffer, length, MSG_NOSIGNAL);Half-Close a TCP Connection
Section titled “Half-Close a TCP Connection”shutdown(fd, SHUT_WR); // No more sends; still receive peer responseshutdown(fd, SHUT_RD); // Stop receivingshutdown(fd, SHUT_RDWR);close(fd); // Release the descriptor after useA request/response client can send its request, call shutdown(fd, SHUT_WR), and continue reading until the server closes its response stream.
10. Useful Socket Options
Section titled “10. Useful Socket Options”int yes = 1;setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof yes);| Option | Level | Use |
|---|---|---|
SO_REUSEADDR | SOL_SOCKET | Rebind a recently used listening address during restart |
SO_REUSEPORT | SOL_SOCKET | Platform-specific load distribution among listeners |
SO_KEEPALIVE | SOL_SOCKET | Detect some dead long-lived TCP peers over time |
SO_RCVTIMEO / SO_SNDTIMEO | SOL_SOCKET | Blocking receive/send timeouts |
SO_RCVBUF / SO_SNDBUF | SOL_SOCKET | Kernel buffer sizing hint |
TCP_NODELAY | IPPROTO_TCP | Disable Nagle delay for latency-sensitive small writes |
IPV6_V6ONLY | IPPROTO_IPV6 | Decide whether an IPv6 listener accepts mapped IPv4 traffic |
SO_BROADCAST | SOL_SOCKET | Permit IPv4 UDP broadcasts |
Options vary across operating systems. Check failures and document why each non-default setting is needed.
Set a Receive Timeout
Section titled “Set a Receive Timeout”struct timeval timeout = { .tv_sec = 5, .tv_usec = 0};if (setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &timeout, sizeof timeout) < 0) { perror("SO_RCVTIMEO");}11. Blocking, Nonblocking, and Multiplexed I/O
Section titled “11. Blocking, Nonblocking, and Multiplexed I/O”Blocking Mode
Section titled “Blocking Mode”Default sockets block the current thread until an operation can progress. It is simple and often enough for small tools or one-thread-per-client servers.
Enable Nonblocking Mode
Section titled “Enable Nonblocking Mode”int flags = fcntl(fd, F_GETFL, 0);if (flags < 0 || fcntl(fd, F_SETFL, flags | O_NONBLOCK) < 0) { perror("fcntl");}With a nonblocking socket:
accept(),recv(), andsend()can fail withEAGAIN/EWOULDBLOCK; wait for readiness and try later.- A nonblocking
connect()often returns-1witherrno == EINPROGRESS; wait for writable readiness and inspectSO_ERROR. - Keep per-connection input/output buffers because reads and writes may be partial.
select, poll, and epoll/kqueue
Section titled “select, poll, and epoll/kqueue”| Mechanism | Best Fit | Notes |
|---|---|---|
select() | Learning and very small descriptor sets | Descriptor limit; rewrites sets each call |
poll() | Portable moderate-size servers | Array scanned on each wait |
epoll | Many Linux connections | Linux-specific readiness API |
kqueue | BSD/macOS event loops | Platform-specific event API |
Small poll() Loop Skeleton
Section titled “Small poll() Loop Skeleton”struct pollfd clients[128] = { { .fd = server_fd, .events = POLLIN }};nfds_t used = 1;
for (;;) { int ready = poll(clients, used, -1); if (ready < 0) { if (errno == EINTR) continue; perror("poll"); break; }
if (clients[0].revents & POLLIN) { int client_fd = accept(server_fd, NULL, NULL); if (client_fd >= 0 && used < 128) { clients[used++] = (struct pollfd){ .fd = client_fd, .events = POLLIN }; } else if (client_fd >= 0) { close(client_fd); } }
for (nfds_t i = 1; i < used; ++i) { if (clients[i].revents & (POLLERR | POLLHUP | POLLNVAL)) { close(clients[i].fd); clients[i] = clients[--used]; --i; continue; } if (clients[i].revents & POLLIN) { char buffer[4096]; ssize_t n = recv(clients[i].fd, buffer, sizeof buffer, 0); if (n <= 0) { close(clients[i].fd); clients[i] = clients[--used]; --i; } else { /* Parse input and enqueue output; handle partial sends. */ } } }}Production event loops must also watch for writable readiness when queued output cannot be sent immediately.
12. Server Designs and Concurrency
Section titled “12. Server Designs and Concurrency”| Design | Advantages | Limits |
|---|---|---|
| Handle one client at a time | Minimal code | All other clients wait |
Process per connection (fork) | Isolation; classic Unix approach | Process overhead |
| Thread per connection | Simple blocking code | Thread/resource limits |
| Thread pool | Bounded concurrency | Needs work queue and backpressure |
| Event loop | Scales to many mostly idle connections | More state management |
| Event loop plus worker pool | Scalable I/O with CPU work offloaded | More coordination |
Robust Server Checklist
Section titled “Robust Server Checklist”- Set limits: maximum clients, maximum frame length, idle timeout, request deadline, and outbound queue size.
- Apply backpressure: do not read endlessly when a slow client cannot receive responses.
- Do not let one client block the accept loop or other connections.
- Handle
SIGTERM/SIGINTso listening sockets and in-flight work can close cleanly. - Log peer endpoint, errors, timeouts, rejected oversized frames, and resource exhaustion.
13. Hostname Resolution and Binding Patterns
Section titled “13. Hostname Resolution and Binding Patterns”Resolve a Client Destination
Section titled “Resolve a Client Destination”struct addrinfo hints = {0};hints.ai_family = AF_UNSPEC;hints.ai_socktype = SOCK_STREAM;
struct addrinfo *results = NULL;int error = getaddrinfo("example.com", "443", &hints, &results);if (error != 0) { fprintf(stderr, "%s\n", gai_strerror(error));}/* Try each result with socket() and connect(). */freeaddrinfo(results);Build a Passive Listener Address
Section titled “Build a Passive Listener Address”struct addrinfo hints = {0};hints.ai_family = AF_UNSPEC;hints.ai_socktype = SOCK_STREAM;hints.ai_flags = AI_PASSIVE; // NULL hostname means wildcard local address
struct addrinfo *addresses = NULL;int error = getaddrinfo(NULL, "8080", &hints, &addresses);Try each returned address until socket() and bind() succeed. This is more portable than assuming only IPv4 is available.
Binding Choices
Section titled “Binding Choices”| Bind Target | Reachability |
|---|---|
127.0.0.1 / ::1 | Only programs on the same host |
| LAN/interface address | Networks that can reach that interface |
0.0.0.0 / :: | All matching local interfaces; protect with firewall/authentication |
14. Unix Domain Sockets
Section titled “14. Unix Domain Sockets”Unix domain sockets (AF_UNIX) communicate between processes on one machine. They avoid network exposure and can use filesystem permissions.
#include <sys/socket.h>#include <sys/un.h>
int fd = socket(AF_UNIX, SOCK_STREAM, 0);struct sockaddr_un addr = {0};addr.sun_family = AF_UNIX;strncpy(addr.sun_path, "/tmp/my-service.sock", sizeof addr.sun_path - 1);
unlink(addr.sun_path); // Remove stale path before server bindbind(fd, (struct sockaddr *)&addr, sizeof addr);listen(fd, 16);
/* accept(), send(), and recv() work as for TCP. *//* On server shutdown: close(fd); unlink(addr.sun_path); */Use a directory with appropriate permissions for sensitive local services. Avoid predictable writable public locations when another user could create the pathname first.
15. Broadcast and Multicast with UDP
Section titled “15. Broadcast and Multicast with UDP”IPv4 Broadcast
Section titled “IPv4 Broadcast”int yes = 1;setsockopt(fd, SOL_SOCKET, SO_BROADCAST, &yes, sizeof yes);/* sendto() a subnet broadcast address only when the network permits it. */Join an IPv4 Multicast Group
Section titled “Join an IPv4 Multicast Group”struct ip_mreq group = {0};inet_pton(AF_INET, "239.0.0.1", &group.imr_multiaddr);group.imr_interface.s_addr = htonl(INADDR_ANY);
if (setsockopt(fd, IPPROTO_IP, IP_ADD_MEMBERSHIP, &group, sizeof group) < 0) { perror("IP_ADD_MEMBERSHIP");}Broadcast and multicast are generally local-network tools, may be restricted by routers or cloud networks, and require care to avoid amplification or unintended data exposure.
16. Protocol Design: From Demo to Real Service
Section titled “16. Protocol Design: From Demo to Real Service”A Small Binary Frame
Section titled “A Small Binary Frame”+----------+----------+----------+-----------------+| version | type | length | payload || 1 byte | 1 byte | 4 bytes | length bytes |+----------+----------+----------+-----------------+For any protocol you design:
- Specify framing, encoding, byte order, valid message types, and protocol version.
- Cap payload size and nesting before parsing or allocating memory.
- Decide what happens on malformed input: error reply, connection close, or ignored UDP datagram.
- Add authentication and authorization when data or actions are sensitive.
- Use request IDs when multiple requests may be in flight or UDP retries may duplicate work.
- Use a standard serialization format when interoperability matters.
Text Protocol Example
Section titled “Text Protocol Example”GET user:42\nOK 5\nAliceEven a text protocol needs limits: maximum line length, permitted commands, allowed encoding, and clear response termination.
17. Security and Reliability Checklist
Section titled “17. Security and Reliability Checklist”| Risk | Defensive Practice |
|---|---|
| Plaintext snooping or tampering | Use TLS through a mature TLS library or terminate TLS at a trusted proxy |
| Buffer overflow | Pass buffer sizes; never assume inbound bytes are null-terminated |
| Memory exhaustion | Set message and queue limits before allocating |
| Slow clients / slowloris | Apply read, write, handshake, and idle timeouts |
| Connection floods | Bound concurrency; apply rate limiting and OS/firewall controls |
| Command injection | Parse input; never concatenate network data into shell commands |
| Unauthorized access | Authenticate clients and authorize operations |
| Untrusted deserialization | Validate types, lengths, ranges, and protocol state |
| Descriptor leaks | Close sockets on every error and shutdown path |
| Public accidental exposure | Bind to loopback during development unless remote access is intended |
Raw TCP does not encrypt or authenticate traffic. TLS is a separate layer, commonly implemented with OpenSSL, wolfSSL, mbed TLS, platform APIs, or a reverse proxy.
18. Debugging and Inspection Commands
Section titled “18. Debugging and Inspection Commands”ss -ltnp # Listening TCP sockets and owning processesss -lunp # Listening UDP socketsnc -vz 127.0.0.1 8080 # Check whether TCP connection succeedsnc 127.0.0.1 8080 # Interact with a TCP text serviceprintf 'ping' | nc -u 127.0.0.1 8080curl -v http://127.0.0.1:8080/tcpdump -nn -i lo port 8080 # Inspect loopback traffic (often requires privileges)strace -e trace=network ./tcp_clientCross-Platform Ideas
Section titled “Cross-Platform Ideas”| Symptom | Check |
|---|---|
bind: Address already in use | Existing listener, wrong address/port, or restart without SO_REUSEADDR |
connect: Connection refused | No listener on destination or firewall rejection |
| Client hangs | Missing message framing, missing response, timeout absent, or waiting for EOF |
| Server receives incomplete text | TCP partial read; accumulate and parse frames |
| Works on localhost only | Bind address, firewall, routing, NAT, or service exposure policy |
| Unexpected disconnect | Peer close, timeout, reset, protocol violation, or process termination from SIGPIPE |
19. Windows Winsock Differences
Section titled “19. Windows Winsock Differences”The socket model is similar, but Windows needs initialization and uses different cleanup/error functions.
#include <winsock2.h>#include <ws2tcpip.h>
WSADATA data;if (WSAStartup(MAKEWORD(2, 2), &data) != 0) { // Handle failure.}
SOCKET fd = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);if (fd == INVALID_SOCKET) { int error = WSAGetLastError();}
closesocket(fd);WSACleanup();| POSIX | Windows |
|---|---|
int fd | SOCKET fd |
Failure is -1 | Failure is INVALID_SOCKET or SOCKET_ERROR |
close(fd) | closesocket(fd) |
errno | WSAGetLastError() |
fcntl(... O_NONBLOCK) | ioctlsocket(... FIONBIO ...) |
| Link normally | Link with Ws2_32.lib / -lws2_32 |
20. C++ RAII Socket Wrapper
Section titled “20. C++ RAII Socket Wrapper”C++ should give socket descriptors ownership semantics so exceptions and early returns do not leak them.
#include <unistd.h>#include <utility>
class Socket {public: explicit Socket(int fd = -1) noexcept : fd_(fd) {} ~Socket() { reset(); }
Socket(const Socket&) = delete; Socket& operator=(const Socket&) = delete;
Socket(Socket&& other) noexcept : fd_(std::exchange(other.fd_, -1)) {}
Socket& operator=(Socket&& other) noexcept { if (this != &other) { reset(std::exchange(other.fd_, -1)); } return *this; }
int get() const noexcept { return fd_; } explicit operator bool() const noexcept { return fd_ >= 0; }
int release() noexcept { return std::exchange(fd_, -1); }
void reset(int next = -1) noexcept { if (fd_ >= 0) { ::close(fd_); } fd_ = next; }
private: int fd_;};On Windows, replace int, -1, and ::close() with SOCKET, INVALID_SOCKET, and ::closesocket(), or wrap platform details behind a small portability layer.
21. Advanced TCP Notes
Section titled “21. Advanced TCP Notes”| Topic | Practical Meaning |
|---|---|
| Three-way handshake | TCP connection setup costs a round trip before application data in ordinary TCP |
TIME_WAIT | A recently closed endpoint may remain recorded to protect later connections from old packets |
| Nagle algorithm | Can batch small writes; TCP_NODELAY may reduce latency at a throughput cost |
| Keepalive | Detects some dead idle peers eventually; it is not an application request deadline |
| Backpressure | A slow receiver eventually prevents unbounded sending; your application must bound queued data |
Reset (RST) | Abrupt termination; pending data may be lost |
| Half-close | One direction may finish while the other still sends data |
Do not assume one request per connection unless your protocol says so. Many protocols keep connections alive and carry multiple framed requests.
22. Advanced UDP Notes
Section titled “22. Advanced UDP Notes”| Topic | Practical Meaning |
|---|---|
| No delivery guarantee | Add retries only when duplicate effects are safe or detectable |
| No ordering guarantee | Carry sequence numbers when order matters |
| No built-in congestion control | Avoid uncontrolled high-rate sending |
| MTU and fragmentation | Oversized datagrams are less reliable across real networks |
| Source spoofing | Do not treat an unauthenticated source address as identity |
| Amplification | Keep unauthenticated responses limited relative to requests |
For reliable encrypted streams over the internet, choose TCP plus TLS or a mature QUIC implementation instead of inventing reliability or cryptography on raw UDP.
23. Learning Path and Quick Recall
Section titled “23. Learning Path and Quick Recall”Beginner
Section titled “Beginner”- Learn address, port, TCP, UDP, and client/server terminology.
- Compile and run the TCP echo server and client on loopback.
- Modify the message and print peer addresses.
- Run the UDP pair and observe the difference in API flow.
Intermediate
Section titled “Intermediate”- Add newline or length-prefixed message framing.
- Support multiple clients using threads or
poll(). - Add timeouts, error handling, size limits, and clean shutdown.
- Switch listeners and clients to
getaddrinfo()for IPv4/IPv6 portability.
Advanced
Section titled “Advanced”- Use nonblocking I/O with per-connection buffers and backpressure.
- Build an event-loop server with well-defined protocol state.
- Add TLS, authentication, rate limits, observability, and resource limits.
- Test malformed inputs, slow clients, disconnects, partial messages, and overload behavior.
API Recall Card
Section titled “API Recall Card”TCP server: socket -> setsockopt -> bind -> listen -> accept -> recv/send -> closeTCP client: socket -> connect -> send/recv -> shutdown/closeUDP server: socket -> bind -> recvfrom/sendto -> closeUDP client: socket -> sendto/recvfrom -> close
Portable addressing: getaddrinfo -> try returned addresses -> freeaddrinfoNever forget: check errors, loop for partial I/O, frame TCP messages, set limits.