[Internet peering] Ratios are an artifact of the '90s

(Quote from Justin M. Streiner in a good general comment he made on all this on the North American Network Operators’ Group (NANOG) mailing list.)

This is an inital reply to Neil Stevens’s Comcast did the right and fair thing cutting off Level 3. The key detail I focus on is the concept of “transit” and how in the 21st century “fair” peering is not necessarily symmetrical, that is with equal traffic (“bandwidth”) going both ways.

There’s a significant difference between Cogent and Comcast: the former is a classic backbone while for the purposes of this discussion I’m treating the latter as a residential broadband ISP (they do some backbone business but I don’t think that’s a factor here except perhaps historically).

In the 20th century backbones traditionally peered with each other on a basis of roughly equal traffic, ideally a ratio of 1:1, because otherwise one of them would be using the other to obtain transit of packets to 3rd parties. I.e. if backbone A wants to communicate with backbone C without settlement costs, it should peer with C, not use B as transit to it without paying, i.e not A<->B<->C but A<->C. As I understand it that was the issue between L3 and Cogent, both backbones. See a railroad analogy in these comments to Neil’s posting.

However Comcast is an entirely different beast: as far as I know, the vast bulk of their traffic is to their endpoint customers, some business but the vast majority being residential like you and me.

All the last mile broadband providers like Comcast are resolutely asymmetrical and this is in part due to an engineering/physical issue known as head end noise (which I’ll explain if desired). It also fits with the 20th century residential customer model of consuming content rather than generating it, i.e. we make small requests like “view this web page” or “watch this movie” and in return we get the much larger content.  Compare to our changing the dial to select between radio and TV content providers broadcasting all the time over the airwaves.

For cable companies using shared coax cable to the neighborhood and Data Over Cable Service Interface Specification (DOCSIS) modems that’s an inherent aspect of the architecture, it’s expensive to push bits upstream (e.g. due to head end noise; see the Wikipedia article on DOCSIS, especially the Equipment section, for a bit more). Cable companies pretty much have to install extra cables upstream to get around this engineering/physical limits issue (it’s an issue for DSL but not hardly as bad because each customer starts out with his own cable that goes straight to the telephone company’s equipment).

So by by network architectural design and customer inclination any backbone’s traffic with Comcast will be asymmetrical. And as I understand it, that was reflected in L3 and Comcast’s previous peering agreement (who are, from what I’ve read, the largest companies in the US in each category) and as implied by Comcast, their previous peering arrangement with L3 already allowed for some asymmetry but now L3’s deal with Netflix is increasing it to 5:1 (or more).

Side note: what really counts in all this is peak bandwidth demand; the pipes are always on and at the lowest level there is no difference in cost between their being empty or nearly full. Unfortunately content providers Netflix are obviously most heavily used in the evening and everyone has to build out their networks to accommodate this peak in demand. The only question is who pays for it where and how.

The fact that more bits are going into Comcast’s network than are coming out of it is not “unfair” to anyone, that’s just how things are, a fundamental aspect of early 21st century Internet service. If the Internet didn’t offer so much tasty content, Comcast wouldn’t have so many customers paying them quite a bit of money to get to it. If Comcast didn’t offer so many paying consumers of content, content providers wouldn’t care so much about connecting to Comcast.

In my next item I’ll address the who, where and how.