On Internet and its Reliability

Aldrin Isaac the Bloomberg guy, has a nice perspective of Internet, its role and its behavior:

Networks prior to the Internet were largely closed systems, and the cost of communicating was extraordinarily high.   In those days, the free exchange of ideas at all levels was held back by cost.  On the Internet, for a cost proportional to a desired amount of access bandwidth, one can communicate with a billion others.  This has propelled human achievement forward over the last 20 years.  By way of Metcalfe’s law, the Internet’s value is immeasurably larger than any private network ever will be.

On reliability, maintainability, incentives:

On a well run private network, bandwidth and communications are regulated to deliver an optimal experience, and network issues are addressed more rapidly as all components are managed by a single operator.  The Internet on the other hand is a network of networks wherein network operators do not have sufficient incentive to transport data for which they are not adequately compensated.

With network neutrality and the corresponding lack of QoS on the Internet, ISPs have to maintain significant backbone and peering capacity to ensure other communication continue to function in the presence of these high volume traffic.  However ISP operators have demonstrated that they are much more inclined to provide capacity to their direct customers than they are to those who are not on their network.

On private vs public (Internet) communication:

On the outside, it seems straight-forward enough for businesses to continue to peer directly over private leased lines.  However there is a trend that is putting pressure on this model.  An increasing number of private businesses are leveraging an emerging landscape of SaaS and IaaS services.  This is driving a general acceptance of the Internet as a primary medium for B2B communication.   Private connectivity also comes with the baggage of added capital and operational costs.

For many businesses, Internet-based B2B communication “as is” may be fine for SaaS services such as HR and billing where a temporary loss of service is survivable.  But there are a class of services and communications that are too-important-to-fail for many businesses and even for larger ecosystem such as capital markets.  Reliable infrastructure is a prerequirement for engaging in these services.

In another blog post, he goes on to mention how problems with Internet can be handled without replumbing the whole Internet:

On the importance of IXPs:

IXPs are where the different networks that make up the Internet most often come together.  Without IXP, the Internet would be separate islands of networks.  IXP are, in a sense, the glue of the Internet.  From the early days of the Internet, IXP have been used to simplify connectivity between ISPs resulting in significant cost savings for them.  Without an IXP, an ISP would need to run separate lines and dedicate network ports for each peer ISP with whom it connects.

On the difference between ISP and IXP:

ISP fabrics are intended to reach customers that are spread out over a wide geographic area.  An IXP fabric on the other hand is fully housed within a single data center facility.   In some cases an IXP fabric is simply a single Ethernet switch.  ISP access links use technology needed to span across neighborhoods, while IXP access links are basically ordinary Ethernet cables that run an average of around several dozen meters.  So essentially the distinction between the two is that an ISP is a WAN and an IXP is a LAN.

The advantage of peering at IXP:

Remember that the cost dynamics of operating an IXP are different than an ISP.  At each IXP these business customers can peer with one another and their favorite ISPs for the cost of a LAN access link.

..being at an IXP creates more direct access to more endpoints at a better price point than buying access lines to numerous ISP.

By connecting to IXP’s B2B customers can at least avoid the cost incurred by having private lines to one another. All they need to have is go to an IXP and plug in (but what is the cost of that?)  One more advantage is that business customers use the same access circuits to communicate with other business customers as well as the rest of the Internet. To conclude he says:

As the Internet enables new SaaS and IaaS providers to find success by avoiding the high entrance cost of building a private service delivery network, more businesses are turning to the Internet to access their solution providers of choice. The old Internet connectivity model cannot reliably support the growing use of the Internet for business and so a better connectivity model is needed for a reliable Internet. New opportunities await.

I am looking forward to hear words of wisdom from this guy.


FC vs FCoE

There is a blog post about a flawed study of FC vs FCoE sponsored by Brocade. Some key takeaways:

  • The technology of FCoE is relatively immature when compared to FC, so comparison of FC and FCoE yet may not be that fair.
  • A study should do end-to-end FC and end-to-end FCoE, but not just FCoE as a one hop solution. If end to end testing is not possible – for example, there is no sufficient choice of products at different hops (servers, storage, network), then that itself is a cue that technology is still catching up.

In the end, he quotes:

I’m just really, really anti-FUD. Compete on tech, compete on features, compete on value, compete on price, compete on whatever it is that makes you different. Just don’t do it in a misleading, dishonest way. Respect your customers enough to know they’ll see through blatant misrepresentations, and respect your products enough to let them compete fairly.

Another guy who is apparently used to it, says about the generic nature of these reports:

First, a little bit about these types of reports. They’re pretty standard in the industry, and they’re commissioned by one company to showcase superiority of a product or solution against one or more of their competitors. They can produce some interesting information, but most of the time it’s a case of: “Here’s our product in a best-case scenario versus other products in a mediocre-to-worst case scenario.” No company would release a test showing other products superior to theirs of course, so they’re only released when a particular company comes out on top, or (most likely) the parameters are changed until they do. As such, they’re typically taken with a grain of salt. In certain markets, such as the load balancer market, vendors will make it rain with these reports on a regular basis.

On the confusion between software and hardware FCoE initiators (which is one of the heated debates of this test report):

No matter how you slice it, it’s troubling. On one hand, if they did configure software initiators, they either ignorantly or willfully sabotaged the FCoE results. If they just didn’t understand the basic VIC concept, it means they setup a test without understand the most basic aspects of the Cisco UCS system. We’re talking 101 level stuff, too.  I suspect it’s the later, but since the only configuration of any of the devices they shared was a worthless screenshot of UCS manager, I can’t be sure.


Root and BPDU Guard

This thread has a concise explanation of RootGuard and BPDUGuard:

The BPDU guard feature is designed to allow network designers to keep the active network topology predictable. BPDU guard is used to protect the switched network from the problems that may be caused by the receipt of BPDUs on ports that should not be receiving them. The receipt of unexpected BPDUs may be accidental or may be part of an unauthorized attempt to add a switch to the network. BPDU guard is best deployed toward user-facing ports to prevent rogue switch network extensions by an attacker.

The root guard feature of Cisco switches is designed to provide a way to enforce the placement of root bridges in the network. Root guard limits the switch ports out of which the root bridge may be negotiated. If a root-guard-enabled port receives BPDUs that are superior to those that the current root bridge is sending, then that port is moved to a root-inconsistent state, which is effectively equal to an STP listening state, and no data traffic is forwarded across that port.