Basics of the Fastly network

Some basics are needed to understand everything else.

First, the Fastly network is made up of "Points-Of-Presence" or "POPs", which is what you might traditionally think of as "one of a CDN's datacenters". (The POP terminology is used throughout all official documentation, so get used to it.)

POPs come in varying flavors and colors and get distributed around the world, and each pop has some number of servers (nodes) inside it. Here's the current global network map. The details of POP routing and networking are magical and not in scope here: we are only concerned with the "topographical map" of the system.

A Fastly POP has some important invariants to keep in mind at any time:

• All nodes are homogeneous. All nodes participate in all active duties described further; there are no special nodes that handle tasks over other nodes for the purposes of caching. They all cache in memory, on disk, run VCL, and handle requests. A client request can potentially be routed to any node in a POP.
• A request takes, at most, two hops, involving two nodes. Fastly (and Varnish) is designed similar to Nix itself: every cache object has a hash associated with it (its "key"), and when a request comes in, we compute the hash from the request, in order to get the cache object (the "value") to use for the response.

This design allows a client request to be served, at most, by two nodes. When a request comes to any node, it calculates the hash, and checks its own cache. This first node is called the edge node. If the desired object does not exist there, it uses the hash as a kind of "clustering key" in order to a (deterministic) lookup of another node in the POP. It then routes the request to that node, performing a "hop". This second node is called the cache node — it also checks its cache, and responds appropriately. The cache node is the node that is responsible for talking to the origin, in the event of a cache miss.

This ensures that if two clients ask for the same object (yielding the same hash), and they request it from two different servers in the POP, the request will get hopped, and routed to a third server (cache node) that will be the same for both requests. This means the cache node will yield a hit for both requests. If every server simply talked to the origin, this would result in a miss, followed by two requests to the origin.

• Fastly VCL is not only run on one node, but sometimes (often), two nodes. As a consequence of the above, when you write VCL, it is run multiple times, and different parts of your VCL run on different nodes. This has a number of technical consequences (which will not be elaborated on as of now), but is worth spelling out explicitly.

This design above is what we refer to as "node clustering" or "intra-POP clustering".

In the above setup, when two nodes "cluster" to handle a request, they both store a copy of the data, but in different locations. The edge node uses memory, but in contrast, the cache node uses fast disk storage. Because nodes are homogeneous, all POP nodes act as edge nodes and cache nodes at the same time, but for different objects.

A corollary of this is that one object can have many copies in memory. (There will be a copy of the object in memory on every edge node that serves a request in a POP.) But there is only one copy of that object on disk in a POP: the copy that exists on the cache node.