Premium Video

This video is available to premium subscribers only

Unlock Full Access

Every request in a distributed system needs to reach the right destination. Whether you're opening a website, one service is calling another, or an application server is connecting to a database, IP addresses play a key role in making that communication possible.

Every IP packet contains source and destination addresses, which routers use to forward traffic across networks toward its destination.

In this chapter, we'll explore how IP addresses work, how they help devices communicate across different networks, and why understanding IP addressing is important when designing distributed systems.

An IP address is a numerical address assigned to a network interface, such as a server's network card, a laptop's Wi-Fi adapter, a VM, or a container.

When another machine wants to communicate with that server, it sends packets to this address. Every packet carries two addresses:

Routers along the way examine the destination IP and forward the packet toward the appropriate network. Each router only needs to know the next step. It does not need to know the full path from client to server.

But an IP address isn't a permanent identifier for a physical machine. A server can have multiple network interfaces, each with its own address. And those addresses can change when the server moves to another network or its configuration changes. A container may get a new address every time it starts.

So it's more useful to think of an IP address as a network location rather than a machine's permanent identity.

IP Addresses vs MAC Addresses

This is different from a MAC address, which is primarily used to deliver data within a local network, such as from your laptop to your home router. IP addresses let packets travel between different networks.

As a packet crosses routers, the destination MAC address changes at each hop, because each hop is a new local delivery. The destination IP usually stays the same until the packet reaches its target, unless a device such as a NAT gateway or proxy rewrites it.

Today, there are two main versions of IP: IPv4 and IPv6. Let's start with IPv4.

An IPv4 address is 32 bits long. We usually write it as four decimal numbers separated by dots. Each number represents 8 bits, so its value ranges from 0 to 255.

With 32 bits in total, IPv4 has 2^32, or about 4.3 billion, possible addresses. That sounded like a lot when IPv4 was introduced. But as billions of phones, computers, servers, and other devices connected to the internet, the available address space became a limitation.

Not all those addresses can be used publicly, either. Some ranges are reserved for private networks and other purposes.

This shortage helped drive the adoption of private IP addresses and NAT, and eventually IPv6. The regional registries that hand out public IPv4 space ran through their free pools during the 2010s, and IPv4 keeps working today largely because of private addressing, NAT, and careful allocation.

But before we get into those, we need to understand how IP address ranges are organized.

An IP address can be divided into two logical parts:

But how do we know which part of an IP address represents the network and which part represents the host?

That's where CIDR notation comes in. It adds a slash followed by a number at the end of an IP address. This number is called the prefix length, and it tells us how many bits, starting from the left, identify the network.

In 192.168.1.10/24, the /24 means the first 24 bits belong to the network portion, leaving the remaining 8 bits for the host portion.

Routers typically use network prefixes to make forwarding decisions, rather than maintaining a separate route for every individual IP address. A single route for 192.168.1.0/24 covers all 256 addresses in that network.

A simplified routing table might look like this:

When more than one route matches, the router picks the most specific one, the route with the longest prefix. This is called longest prefix match. 0.0.0.0/0 matches every IPv4 address, so it is only used when nothing more specific matches.

By adjusting the prefix length, we can divide larger address blocks into smaller subnets or combine multiple address blocks into larger routing ranges. For example, 192.168.1.0/24 can be split into two halves, 192.168.1.0/25 and 192.168.1.128/25.

Note: Early IPv4 used fixed address "classes" (A, B, and C), where the first bits of an address decided the network size. That wasted a lot of address space, and CIDR replaced it in the 1990s. You may still hear "Class C" in conversation, but modern networks are planned with CIDR.

CIDR stands for Classless Inter-Domain Routing. It's a way to represent blocks of IP addresses using a network prefix and its length.

For 192.168.1.0/24, the 24 tells us that the first 24 bits identify the network. That leaves 8 bits for addresses within the network, giving us 2^8, or 256, addresses.

The important thing to understand is that the longer the prefix length, the smaller the address block. Each extra bit of prefix halves the block.

Keep in mind that these are total address counts, not necessarily the number of addresses available to devices. Some addresses may be reserved for special purposes:

Example: 10.0.0.0/20

The prefix /20 leaves 12 host bits:

You'll encounter CIDR frequently, especially when working with cloud networks, routing tables, firewall rules, and subnet configuration.

A subnet, short for subnetwork, is a smaller network carved out of a larger address range.

Suppose your application has the range 10.0.0.0/16. Instead of putting every resource into one large subnet, you could divide it into smaller ones. For example, one could be for public-facing resources, another for application servers, and another for databases.

Each subnet has its own range and can be configured with different routing and security rules. You might place an internet-facing load balancer in a public subnet, while application servers and databases stay in private subnets.

This makes your network easier to organize and gives you more control over communication between components. For example, the database subnet can accept connections only from the application subnet.

But simply placing resources in separate subnets doesn't secure them. You still need the right routes and security rules.

In cloud environments, these resources usually live inside a virtual network, such as an AWS Virtual Private Cloud, or VPC. Azure calls it a Virtual Network (VNet), and Google Cloud also calls it a VPC.

When you create a VPC, you assign it a CIDR block, such as 10.0.0.0/16, and divide that space into subnets based on your workloads. A public subnet can contain an internet-facing load balancer. Private subnets can hold application servers and databases.

In AWS, a public subnet has a route to an internet gateway. A private subnet doesn't provide that direct route to the internet. The difference shows up in each subnet's route table:

Routing tables, NAT gateways, and security rules determine how resources communicate and which connections are permitted.

Choosing address ranges carefully matters. Two mistakes are especially expensive to fix later:

A few habits help:

Another important distinction is between public and private IP addresses.

A public IP address can be routed over the internet. Public address space is coordinated through regional registries, ISPs, and cloud providers, so unrelated networks don't use the same public ranges.

A private IP address is used within internal networks and cannot be routed directly over the public internet. IPv4 defines three main ranges of private IP addresses:

These ranges are defined in RFC 1918. Private addresses only need to be unique inside the network where they are used. Your laptop and your neighbor's laptop can both be 192.168.1.10 because those addresses live behind different routers.

In a typical application, users send requests to a public-facing load balancer, which forwards them to application servers using private IP addresses. These servers then communicate with a database over the private network.

This setup allows your application servers and database to remain private, without exposing them directly to the internet.

Keep in mind that a public IP only means an address is reachable. Whether it is safe still depends on firewall rules, authentication, and the application itself.

But what happens when an application server needs to call an external API or download a software update? Since its private IP address cannot be routed over the public internet, it needs a way to communicate with external services.

That's where Network Address Translation, or NAT, comes in.

NAT stands for Network Address Translation. It allows devices with private IPv4 addresses to communicate with services on the public internet without needing their own public IP addresses.

Imagine three application servers, each with a different private IP address. When one of these servers sends a request to an external service, the packet first passes through a NAT device.

The translation records might look like this:

When a reply arrives at 203.0.113.20:40001, the NAT device looks up that port and sends the reply to 10.0.2.11.

In cloud environments, NAT gateways are commonly used for this purpose. They allow private servers to access external APIs, download software updates, and communicate with internet services without assigning a public IP address to each server.

NAT also helps conserve the limited IPv4 address space. It is one of the main reasons IPv4 is still usable today. The same idea lets a home router put a phone, a laptop, and a TV behind one public address.

However, NAT itself is not a security mechanism. We still need proper firewall rules and network policies to control which connections are allowed.

NAT also adds some operational concerns:

IP addresses can also be static or dynamic.

A static IP stays the same until someone deliberately changes it. That's useful when another service needs a predictable address. For example, a partner API might allow incoming requests only from a list of approved public IPs. In AWS, an Elastic IP is a common way to get one.

A dynamic IP is assigned automatically and may change over time. On home and office networks, DHCP (Dynamic Host Configuration Protocol) commonly assigns these addresses.

Dynamic addressing is also common in the cloud, where instances are constantly created, replaced, and scaled. A replacement server might receive a completely different IP. Containers and Kubernetes pods change addresses even more often.

That's why we generally avoid hardcoding server IP addresses into applications. Instead, we rely on DNS, load balancers, and service discovery to find services as their underlying machines change.

Most people don't access websites by typing an IP address. Instead, they use domain names like api.example.com.

DNS, or the Domain Name System, translates these names into IP addresses so clients can find the servers they need to connect to.

This also makes it easier to manage services. If a service moves to a different IP address, we can simply update its DNS record instead of changing every client. Clients gradually start using the new address as their cached DNS records expire, which is controlled by each record's TTL (time to live).

In larger systems, a domain name often points to a load balancer rather than an individual server. The load balancer then distributes incoming requests across healthy backend servers.

For communication between internal services, we use a similar concept called service discovery. It allows services to find available instances without needing to know their IP addresses in advance. Kubernetes Services, Consul, and AWS Cloud Map are common examples.

Together, DNS, load balancing, and service discovery allow applications to communicate reliably, even as the servers and their IP addresses change behind the scenes.

One of the main reasons IPv6 was introduced is that IPv4 simply doesn't have enough addresses to support the growing number of devices connected to the internet.

While IPv4 uses 32-bit addresses, IPv6 uses 128-bit addresses, providing a vastly larger address space: about 3.4 × 10^38 addresses.

IPv6 addresses are written in hexadecimal, as eight groups separated by colons:

To make them shorter and easier to read, two rules apply:

The address above can be written as:

With so many addresses available, IPv6 greatly reduces the need for NAT just to conserve IP addresses. Every device can have a globally unique address. Firewalls are still needed to decide which of those devices can be reached.

Today, IPv4 and IPv6 continue to coexist. Many networks and services support both, which is called dual-stack. A dual-stack service publishes an A record for its IPv4 address and an AAAA record for its IPv6 address. Which version gets used depends on the client's network connectivity and the service's configuration. Clients usually try IPv6 when it is available and fall back to IPv4.

Running both means firewall rules, load balancers, logs, and monitoring need to handle both address families.

A few mistakes come up often when systems depend on IP addresses.

Treating an IP as Identity

An IP address can change, be shared through NAT, or belong to a load balancer or proxy instead of one machine. It is a useful signal, but not a reliable user or device identity.

Losing the Client's IP

Once traffic passes through a load balancer or proxy, the source IP the application sees is often the proxy's address. To keep the original client IP for logs, abuse detection, and rate limiting, proxies pass it along in headers such as X-Forwarded-For or through the Proxy Protocol.

Only trust these values when they are set by infrastructure you control. A client can send a fake X-Forwarded-For header unless your proxy overwrites it.

Special Addresses

Some addresses have special meanings that are worth recognizing during debugging:

A common bug: a service bound to 127.0.0.1 works locally but cannot be reached from another machine. Binding to 0.0.0.0 makes it listen on all interfaces.

IP addresses identify network interfaces and let routers forward packets between networks. They describe a network location, not a machine's permanent identity.

CIDR and subnetting help us organize address ranges. The prefix length decides how many bits identify the network, and a longer prefix means a smaller block.

Public and private addressing, routing rules, and NAT determine how resources connect to each other and to the internet. In the cloud, a VPC is split into public and private subnets, and its range should be chosen to avoid overlaps and leave room to grow.

IPv6 removes the address shortage of IPv4, and most services today run both side by side.

And because servers can move, scale, or be replaced, distributed systems usually use DNS, load balancers, and service discovery rather than relying on hardcoded IP addresses. That gives applications stable ways to reach services without tying them to individual machines.