Skip to content

BGP (Border Gateway Protocol)

This course's landing page already named BGP as "the map everyone shares" and flagged that "a misconfigured or malicious announcement can pull traffic toward the wrong place." The anycast article leaned on BGP's path-selection behavior directly, promising the mechanism itself for later. This is that mechanism — the actual protocol that lets autonomous systems tell each other what they can reach, and the reason that trust-based design is both how the internet holds together and how it occasionally breaks in public.

A path-vector protocol: routes carry their own history

Every routing protocol needs some way to decide which of several possible paths to a destination is the "best" one. Protocols used inside a single network — OSPF, for instance — typically do this by exchanging detailed link information and calculating shortest paths mathematically, the way network topologies described connections within one administrative domain. That approach doesn't scale to the internet as a whole: no single entity is allowed to know, or trusted to know, every link and its exact cost across thousands of independently operated networks.

BGP takes a fundamentally different approach, called path-vector routing: instead of exchanging raw link costs, each BGP-speaking router simply tells its neighbors, "I can reach this address block, and here is the exact sequence of autonomous systems the traffic would pass through to get there." That sequence — the AS_PATH — is the single most important piece of information BGP carries, and it does two jobs at once: it lets a receiving network judge how many networks a path crosses (a rough proxy for preferring shorter, more direct routes), and it lets every router trivially detect and reject a routing loop, since an AS that sees its own number already present in an announced AS_PATH knows immediately that accepting it would create a loop.

AS 65010 announces 203.0.113.0/24 to its neighbor AS 65020.
AS 65020 announces this onward to AS 65030, appending itself to the path:

  Original announcement:  AS_PATH = [65010]
  After AS 65020 forwards it:  AS_PATH = [65020, 65010]
  After AS 65030 forwards it:  AS_PATH = [65030, 65020, 65010]

Every hop a route announcement crosses adds one more AS number to the front of this list — which is exactly why the AS_PATH's length is one of the strongest signals BGP has for "how far away is this," even though it says nothing about actual bandwidth, congestion, or physical distance.

UPDATE messages: announcing and withdrawing reachability

BGP routers exchange information through a small set of message types defined in RFC 4271, and the one that matters for understanding how routing actually changes is the UPDATE message. An UPDATE does one of two things: it announces a new or changed route (a prefix, its AS_PATH, and several other attributes), or it explicitly withdraws a route that's no longer valid.

UPDATE message announcing a route:
  NLRI (Network Layer Reachability Information): 203.0.113.0/24
  AS_PATH: 65030, 65020, 65010
  NEXT_HOP: 198.51.100.1

UPDATE message withdrawing a route:
  Withdrawn Routes: 203.0.113.0/24

NLRI is simply the prefix being announced — the block of addresses this route can deliver traffic to. NEXT_HOP tells a receiving router which specific address to actually forward matching traffic toward, distinct from the AS_PATH, which only describes the route's history for loop detection and path comparison. A withdrawal carries no AS_PATH or NEXT_HOP at all — it's purely a statement that a previously announced prefix should no longer be considered reachable via whatever route announced it, and every router that had installed that route needs to remove it and fall back to any other route it might have for the same prefix, or conclude the destination is now unreachable if no alternative exists.

This announce-and-withdraw model is also why BGP convergence after a real network failure isn't instantaneous. A link going down triggers a withdrawal, that withdrawal has to propagate hop by hop across however many autonomous systems had learned the now-invalid route, and each of those routers has to recompute its own best remaining path before traffic reliably stops flowing toward the dead route — the process the landing page already described as taking "seconds to minutes," not the sub-second failover a single, centrally-managed network can sometimes achieve.

How a router picks a best path among several candidates

A router with several BGP neighbors will often learn multiple different routes to the exact same prefix, and BGP needs one deterministic rule for which one to actually use for forwarding. The full decision process defined in RFC 4271 has more than a dozen tie-breaking steps, but the ones that matter for understanding real-world behavior are a short, ordered list:

  1. Highest local preference — an operator-configured value expressing "prefer this path over other paths, regardless of anything else," typically used to prefer a paid transit route or a specific peering arrangement over an alternative.
  2. Shortest AS_PATH — fewer autonomous systems crossed is generally preferred, all else being equal. This is the criterion anycast relies on to steer different clients toward different anycast instances of the same announced address.
  3. Lowest origin type, then other tie-breakers — a series of increasingly specific rules (origin type, MED, eBGP over iBGP, lowest router ID, and others) that only come into play when every earlier criterion produced an exact tie.

The practical upshot: an operator with a specific business reason to prefer one upstream provider over another — a cheaper contract, a peering relationship with better performance — sets local preference to express that choice directly, and BGP will honor it even when a shorter, "closer-looking" AS_PATH exists through the disfavored provider. Path selection in BGP is policy first, topology second — this is precisely why RFC 4786's caution, already noted in the anycast article, holds: what BGP considers the "best" route is shaped by configured preferences and business relationships as much as by anything resembling physical proximity.

Practical scenario: a route that was never authorized to exist

The clearest real-world illustration of what happens when BGP's trust model is abused — even accidentally — is the well-documented February 2008 incident in which Pakistan Telecom (AS17557), acting on a government order to block YouTube domestically within its own network, announced the specific prefix 208.65.153.0/24 — a block belonging to YouTube, not to Pakistan Telecom — intending the announcement to stay contained within its own network.

Before the incident:
  YouTube's legitimate route: AS_PATH = [..., 36561]  (YouTube's own AS)

After Pakistan Telecom's announcement leaked to the wider internet:
  Competing route: AS_PATH = [..., 17557]  (Pakistan Telecom)
  17557's AS_PATH is shorter/more specific from many vantage points

Pakistan Telecom's upstream provider, PCCW Global (AS3491), failed to filter this announcement before forwarding it onward to its own peers, and BGP's path-vector mechanism did exactly what it was designed to do: it propagated a syntactically valid-looking route announcement globally, with no built-in mechanism to ask "does this AS actually have the authority to originate routes for this address block?" Within minutes, a significant share of global internet traffic destined for YouTube was instead being routed toward Pakistan Telecom's network, which had never intended to receive any of it and had no capacity to actually serve it — the traffic simply vanished into a black hole. It took roughly two hours for YouTube's own network operators and PCCW to identify the bogus announcement and for PCCW to withdraw it, restoring the legitimate route worldwide.

The structural lesson matters more than the specific incident: BGP, as originally deployed, has no cryptographic way to verify that an AS announcing a prefix is actually authorized to do so — the protocol trusts every announcement it receives from a configured neighbor by default. The mitigation the industry has converged on is RPKI (Resource Public Key Infrastructure, RFC 6480), which lets an address block's legitimate owner cryptographically sign a Route Origin Authorization (ROA) stating exactly which AS is allowed to originate routes for that block. A network performing RPKI-based origin validation can then automatically reject an announcement like Pakistan Telecom's — one for a prefix it holds no valid ROA for — before it ever gets forwarded onward, rather than relying on a human at an upstream provider to notice and intervene hours after the fact.

Warning

RPKI validation only protects a network that actually enables it, and it only stops origin mis-announcements — a more sophisticated route leak that preserves a plausible-looking but still incorrect AS_PATH requires additional mechanisms this article doesn't cover. Enabling RPKI validation on a production edge router is a routing-table change with the same real risk as any other: confirm current routing behavior first, test in a lab or on a non-critical router before rolling it out broadly, and keep a way to disable validation quickly if it unexpectedly rejects routes the network actually needs.

Practice exercises

  1. Using the AS_PATH mechanism above, explain specifically how a BGP router detects and avoids a routing loop without needing any information beyond the announcement it just received.
  2. A network operator sets a high local preference for routes learned from Provider A, even though Provider B consistently offers a shorter AS_PATH to most destinations. Using the path-selection order above, predict which provider's routes actually get used, and explain why AS_PATH length never even gets evaluated in this case.
  3. Explain, using the Pakistan Telecom incident, what specifically RPKI origin validation would have needed to be true (which party's ROA, checked by which router) to have prevented the leaked announcement from propagating past PCCW.

BGP decides which network path a packet's early hops will take to leave its home network and start crossing the wider internet. Everything this module has covered so far — anycast routing, shaping traffic at the edge, the latency and bandwidth math of a single connection — has assumed that path, once chosen, simply carries the packet from one specific server to another. The final piece this module covers steps back into a data center and asks a narrower, more common question: once traffic legitimately needs to reach any one of several identical backend servers, which specific algorithm decides who actually gets it.

Sources