Modernization of IX Billing

There are a few different types of connectivity. Internet exchanges, or IXs, are a common way for networks to reach a large number of other networks, all via a single router interface. Today I want to discuss the economics of IX connectivity, and how many IXs haven’t kept up with the commercial terms offered in the broader connectivity market.

I’m going to avoid the topic of per-unit costs. This is something that would usually need to be negotiated between the parties involved, and much like transit, there are always a number of unique factors that must be considered. If you’re interested in this topic, Dave Temkin gave a presentation on transit vs IX pricing trends at NANOG 67, and while the numbers have changed, the theme remains the same. (Here are links to Dave’s slides and presentation.)

Rather, I want to focus on the billing models that are commonly used today. For a long time, IXs have preferred to use the flat-rate billing model. Simple enough, you take a port, you pay for that full port. It’s something that IXs like because it gives them guaranteed revenue for each port, with minimal billing effort. Some customers/networks also like it because it gives them a simple and consistent bill each month – to the extent that they can set up automatic or recurring payments.

However, for larger networks, the flat-rate billing structure alone can make IX connectivity extremely cost inefficient. When working in multiple regions, networks frequently buy transit connectivity under a single aggregate commit. This allows buyers to pool their needs from different geographic zones, and lump it into a single line item. This increases the billable value of the service being sold, and makes it easier to obtain a lower unit (per Mbps) cost. As a result, this yields a more favorable financial offering for the buyer.

What’s happening?

Over the past several years, we’ve started to see more content heavy networks minimize or disconnect from IXs. Akamai has a long-standing practice of shedding (or overflowing) traffic from IXs onto transit. (Meaning, if you’re an eyeball network, and share a mutual IX with Akamai, they may choose to send you content via transit, instead of the “more direct” path offered over the IX.) Microsoft has developed a similar practice in recent years.

More recently, we’ve started to see Google withdraw completely from some IXs. In other cases Google is refusing to add new bilateral sessions, and has depeered route servers, which means multilateral peering is also no longer available for Google’s routes. As a result networks aren’t able to add new IX peering with Google, and Google is pushing many of these networks to use “VPP” (Verified Peering Providers, which are frequently transit providers) to reach them, or PNIs if traffic levels warrant.

If peering is supposed to be better than transit, then why would networks do this?

In major cities, the effective performance of transit is equal to peering. There’s no added latency, and transit frequently bills using a burstable model, commonly 95th percentile. This allows larger networks to pay based on their usage, not their available capacity.

This creates trouble for IXs. If less content remains available via the IX, then eyeball networks inherently receive less value from the IX. If eyeball networks were to also leave IXs, then a downward spiral could begin.

But there is a way to save the IXs!

If IXs begin to adopt burstable billing, just as transit did, this could very well incentivize larger networks to build out that extra capacity, and IXs would likely begin to see an increase in traffic. While burstable billing does add some overhead to the IX operators, it’s minimal. There are plenty of free and open-source software packages that can assist with such billing.

More and more network operators have started to ask the same thing of IX operators. Why are IXs largely hesitant to adopt the change? How long will it take for them to give in?