Internet Socialism: All-I-can-eat mentality

Every few months, my good friend Jeremy finds a reason to write another post against bandwidth throttling and usage-based billing. Unfortunately, all the blog posts of this world will not change the basic fact (sometimes known as the first law of thermodynamics): there is no free lunch. Applied to this particular issue:

  • Any form of fixed Internet pricing is effectively an “all-you-can-eat” buffet. Such a buffet works as long as the visitors’ stomach sizes have comparable capacity. In the high-speed Internet world a torrent user can consume two or three orders of magnitude more resources than a regular user.
  • In an environment where a minority of users consumes most of the resources, you’re simply forced to treat the large consumers differently. Otherwise, you’re forcing the majority to pay for the excesses of the few and the majority will eventually revolt (which is why the big socialist experiments didn’t work).
  • Obviously you need to upgrade your network as the average use increases, but being forced to upgrade due to a few large consumers and distributing the costs across the whole customer base simply does not make sense (not to mention the fact that providing Internet connectivity is far away from being a lucrative business).

Unfortunately, the basic facts are usually obscured by controversies like companies choosing PR disasters over fixing their networks or Service Providers incompetent enough to call port scanning a DOS attack. It’s also highly unreasonable to expect the users to consume less than 5GB a month and charge any over-the-quota traffic without any safeguards.

There are technical solutions (for example, Cisco’s SCE) that allow the Service Providers to give each user a fair share of the bandwidth (or even limit the number of TCP/UDP sessions in a time period). However, without end-users and bloggers adopting a realistic view of the world, we’re facing a lose-lose scenario. Whatever the Service Providers do, however much they might invest in their network (and charge everyone), however reasonable their throttling/capping decisions might be, the all-I-can-eat zealots will cry foul … or am I yet again completely wrong?

26 comments:

  1. The problem is that you're being sold a 50Mbps connection, and are only able to actually use an average of 2Kbps. The "whiners" are people who work out the math and complain that the companies involved sell them one thing, then radically change the service when you actually try to use what they sold you. Regardless of whatever claimed necessity there is, that is a classic bait-and-switch on the part of the provider. Most consumers won't notice, and for most consumers there is no tangible difference, but that doesn't change the fact that the SP is trying to dramatically reduce your service in response to you actually using the service you paid for.

    If this were being sold as a dedicated circuit, it would have to be billed as "5GB/mo" or "2Kbps, burstable to 50Mbps", which are vastly different from the "50mbps" you originally bought.

    And you have the socialism vs. capitalism thing exactly backwards, by the way, which also detracts from your argument.

    ReplyDelete
  2. James, thanks for the comment. I absolutely agree that the SP service definitions are intentionally misleading (I might write more about that after seeing a few more comments :).

    However, where do you think I went wrong with the socialism vs. capitalism argument?

    ReplyDelete
  3. Once again I read another apologist post somehow claiming that the motivation for these new caps is somehow based on hardware costs and network expenses. To date not a single ISP that is proposing these caps has been able to justify this position. Comparisons to other utilities like electricity fail to take into account that these generate the product and THEN transmit it. If an ISP was saying “Hey we spend millions of dollars reporting the news and making new games and we need to get paid back” then the comparison would be valid. An ISP is in the transmission business and nothing more.

    The real reason that these caps are being put in place is to prevent services like Netflix from directly competing with other services that ISPs provide. COX does not want you using Netflix on line since it eats into their profits from their cable services. Of course these ISPs are going to blame torrent users for the problems since all torrent users are evil child molesting, terrorist sympathizing, pirates and must be doing something we need to stop. Then again no ISP has been able to provide read documented details on how much of their bandwidth is being used by torrent services compared to other usages.

    ISPs are making claims to try and justify their decisions so lock out competitors from the internet at the same time hiding any data that would support their numbers. Until we have complete transparency into the ISP that proves that usage caps are motivated solely by business needs of the ISP and not based on a desire to protect other parts of their business that cannot be tolerated.

    ReplyDelete
  4. No one's looking for "all-you-can eat" service. One expects his connection to be bound by the throughput limits as advertised in Mbps (a far more granular and technically useful metric than measuring transfer by the month); transfer caps are a *further* restriction being placed on what is already explicitly limited. Any need to treat differntly any two customers (one "light" and one "heavy") subscribing to the same service plan is plainly a symptom of gross overprovisioning. Overprovisioning is of course an element in any network design, but many providers have pushed the tolerances too far and it's catching up to them.

    Additionally, it can't be conjectured that Internet service provision is an inherently unprofitable industry because one or several have failed to turn a profit in recent years. This would be akin to arguing that auto manufacturing is unprofitable because GM went bankrupt while ignoring the successes of Toyota.

    Unfortunately, those entrepeneurs who might have better ideas on how run an ISP face a staggering barrier to market entry. It seems that, despite their supposed inability to turn a profit, few companies are willing to relinquish their monopoly on a market or medium.

    ReplyDelete
  5. @Kurt: I'm sorry you view my text as an "apologetic post". I have no interest whatsoever in defending individual ISPs, I'm just worried about the direction into which Internet is moving (and I'm definitely not alone). Getting a reasonable perspective on end-user expectations is just one small bit of the puzzle.

    @Stretch: are you trying to persuade me that you actually believe you can get 20M 24x7 downstream transfer rate for $54.99? I know you better than that :))

    Also, I agree with the GM vs. Toyota argument. We just need to find the Toyota. Any ideas?

    Last but not least, what's stopping a startup from pulling fiber to every home in Beaumont,TX? Are you telling me that in the most market-oriented country in the world you cannot install a fiber network if you have the finance and means to do so?

    I simply cannot believe that; there are at least two Service Providers installing FTTH in the backwater Ljubljana, Slovenia (population: 270.000) and I have a symmetrical 20/20 FTTH service in the middle of nowhere less than 100 miles from Balkan.

    ReplyDelete
  6. Ivan (way, way OT),

    The greedy few are unfairly exploiting the commonly held resources at the expense of the great un-torrented masses? Arguing in favor of the "commons" against the individuals capable of putting them to selfish use?

    Both are left-wing ideals. The fact that a ubiquitous high-bandwidth last-mile exists is simply lowering the bar to entry for the "exploiting class" :-)

    ReplyDelete
  7. @James: it's not off-topic, I never viewed this issue from your perspective. Thanks for sharing it, I have to agree with your conclusions (whether I like them or not :) ... and I thought I was leaning too far to the right ;)

    BTW, I was not arguing in favor of the "commons", I'm just saying that it's unreasonable to expect that the administrator of the common resources (Service Provider) will allow the "greedy few" to continue with their behavior when it impacts the commonly held resources.

    The alternative would be to view the whining of the greedy when they get a slap on the hand by the administrator as part of the game, but I think this is dangerous, as whining repeated often enough (and broadcasted by media/bloggers) eventually becomes accepted "truth". We need to start exercising our common sense or we'll bring down the whole system (like the bankers almost managed to do).

    ReplyDelete
  8. Ivan, haven't been here for quite a while and although I've disagreed with you before, you have hit the nail on the head.
    The economies of scale become all to obvious when you find yourself on the end of 10000 kms of submarine cable and your major cities are divided by vast distances and years of regulatory mismanagement. (I doubt the rural backwaters of any major country is any different!)
    However, the biggest abuser of bandwidth is not P2P but Microsoft with their poor and bloated (non-optimized) architecture for distributing Windows updates.

    ReplyDelete
  9. @Ivan: Yes. Why *wouldn't* you expect to receive what you've paid for? Cynicism in this regard only works against you as a consumer. While it may have seemed implausible five or ten years ago, such service is available today as a result of technological evolution.

    And before you counter that it isn't available, I'll mention that I have no issue saturating my 12/2 from Cox ($44/mo) at length. Obviously I'm not running torrents 24/7 just to burn bits, but it's always there, as I've paid for it to be there.

    You can't claim that a costomer is acting out of greed when he fairly consumes a service for which he has paid. If a user who has paid for a 5 Mbps connection began (somehow) using 10 Mbps, *that* would be unfair. You cannot fault him for using 5 Mbps, however.

    What's happening is the reverse: a service provider is telling that customer he can only use (for instance) 1 Mbps of the 5 Mbps he has paid for. So where's the greed?

    Additionally, the simple fact that there *are* Internet service providers is evidence enough that *someone* sees a profit to be made. Where there's demand, someone will supply it for a profit. I'm sure you don't expect that they would operate at a loss out of the goodness of their hearts.

    Regarding the lack of FTTH competition, I think you underestimate the influence of US telcos; some even have the balls to sue municipalities for deploying their own fiber networks: http://arstechnica.com/old/content/2008/09/telco-to-town-were-suing-you-because-we-care.ars (BTW, it seems copy & paste doesn't work on the comment form.)

    Additionally, this idea simply doesn't scale. It makes no sense (and in many respects, is not physically possible) for multiple providers to each maintain an independent cable plant in parallel with the others. A much more elegant solution would be to entrust the maintenance of a common physical infrastructure to a local utility or third party. This leaves the choice of provider to the consumer, and benefits providers by significantly lowering their operating costs.

    ReplyDelete
  10. Jeremy,

    * the "greedy few" was not my wording, I've just used it in my reply, and even then in quotes.

    * the argument "I'm paying for 12/2, so I shall have it" is valid (and you would be right to object if you'd constantly fail to get that rate) as long as you don't overuse it and impact other users by doing so, which was (supposed to be) the point of my article. That's why I've written "believe you can get 20M 24x7 downstream transfer rate". We both know what we're talking about and playing with words won't get us anywhere.

    * on the profits side, I absolutely agree with you that there's obviously some profit to be made, but the question is what the profit margin is (low) and whether the profit allows you to invest in upgrades (questionable).

    * FTTH competition: we agree (and we're not alone) it would make sense to declare the local dark fiber a "public utility" and charge it like you get charged for garbage collection. Will it happen? One can only hope, it's in my opinion one of the few reasonable ways forward. Will there be opposition to it? Absolutely, from any SP who has existing infrastructure and would like to retain its somewhat-monopolistic status and prevent competition.

    ReplyDelete
  11. It's just too good a business model to advertise the fastest possible internet, accept all the incoming customers, then throw or pester out the torrenters, or just plain sabotage their connection.

    That way, profits *can* be made. I'm sure you will find out in this discussion that nothing, nothing whatsoever will satisfy the "I want to torrent at 50mbps every day" crowd.

    Also you said that there are ways to fairly distribute bandwidth ? I know there are on a single router, but do you have any such protocol that operates acceptably over a whole network ? Say I have 20 lexes, all of them having some 200 customers who expect fast http downloads, each lex connected at 200 mbit. 5 of them have 4 50mbps torrenters each. I'm a well-connected ISP, having 2 gigabit transit connections, and connected two, oh, 2 internet exchange points, again 1 gbps each. One can use RSVP to reserve bandwidth for business users (who pay for that).

    This would represent an enormous investment, since incoming and "outgoing" bandwidth is about the same (4 gbps each). Oversubscription is not the traditional 20 or 30, but merely a factor 10, which is not cost-realistic. How do I make sure torrenters don't eat up my transit connections, or lex interlinks, whole ?

    ReplyDelete
  12. @Ivan: Where does this notion that service providers operate on slim margins stem from? I agree that it's the case in the wholesale market but residential access is an entirely different world. As most providers enjoy a monopoly on their medium (if not the entire geographic market), it's obvious that they can pad their rates. Open competition seems to almost linearly reduce prices: http://static.arstechnica.com/2009-broadband-4.png

    @Anonymous: The language you use here seems to betray a resentment for customers who demand what they've paid for. If you offer n Mbps service, expect customers to consume n Mpbs service. If you can't deal with that, don't offer the service, or make room for someone who can.

    ReplyDelete
  13. @stretch

    I hope you can understand that the competition can't fulfill their promises either. And promising less is suicide, I'm sure you can appreciate that.

    So you promise the full ADSL2+ bandwidth, or the full DOCSIS 3.0 bandwidth, as applies to your technology.

    That's the bandwidth the customer *will* have into your network btw. And there is NO way to guarantee connection to specific networks, so we simply don't make claims about that (unless you pay specifically for that), and don't provide it either. 50 mbps means 50 mbps into the lex, nothing more. Of course if you as a customer choose to read the impossible, that 50 mbps advertised means an ISP has x*50mbps to any possible destination, then please keep dreaming, join the communist party and bomb innocent people for utopia, like any good socialist does.

    Say I sell you a business 100 mpbs service. On an exchange I peer with the network you want to communicate with, but they've only got a 10mbps connection on that exchange. Then you'll get 10 mbps only, nothing I can do about that.

    A while ago we had a problem with sparkle (Italy Telecom). Their peering connection was full and they refused to upgrade, citing costs. What, exactly should I do about that (the problem still exists, so feel free to suggest tactics).

    Internet does not have unlimited bandwidth, 1 gbps is still a lot, and the absolute maximum bandwith I have to any one destination. So what do you suggest I do ? Create a nationwide 2-gigabit network to connect no more than 50 20mbps customers, because that's all I can guarantee ? And I still am not able to guarantee that those 1 gbps is not filled up by incoming traffic, either in a direct ddos attack or simply someone who's scp'in the next version of ubuntu, or the new harry potter movie.

    If you're going to complain about "users not getting what they paid for", you might want to give 5 seconds thought exactly how unrealistic you're being.

    I happen to have a degree in informatics, and with graph theory you can easily predict the necessary international bandwidth to guarantee that every network world-wide can talk to eachother with 1 kbps in a worst-case scenario. You know how much it is ? A bit over 20 Terabits per second. Know any network that has more than 10 gbit ? Would you perhaps prefer all peering connections to be limited to 1kbps, until international backbones like verizon, level3, cogent, etc. upgrade to 20 terabit network interfaces ? Then Isp's would at least have the option of "delivering what they advertise". Of course it would also mean that you max. torrent speed is 1 kbps / number of subscribers. On comcast, that'd be what ? One bit per week ?

    ReplyDelete
  14. @Anonymous: Wow. Fail.

    Which provider is this that you work for?

    ReplyDelete
  15. Pffff I've learned long ago to post anonymousely when being honest about these things.

    Anyway ... admin : how do you do cross-router fair-queueing ? Qos so that all clients "tend" to use the same bandwidth (preferably both incoming and outgoing traffic, because the alternative is throttling "all sorts of protocols" ? (@stretch : guess which ones :-p)

    ReplyDelete
  16. Our SCE gurus tell me Service Control Engine from Cisco (and probably several competitive devices)allows you to evenly distribute whatever bandwidth is available to a service group between the subscribers. It can also limit the number of TCP or UDP sessions a subscriber can initiate in a time period. If you're interested in more details, contact me out-of-band (using the information provided in the "Contact me" page reachable from the top menu bar).

    ReplyDelete
  17. @Ivan,

    I think the wrinkle is that the service providers are commanded to run a budget surplus for a group of stakeholders other than their customers, whereas the traditional social model involves an implicitly non-profit organization who's charters typically mention inconcrete objectives like "securing the general welfare."

    ReplyDelete
  18. Jeremy, am I understanding correctly that you advocate that a home user paying, say EUR10, be treated the same as a business user paying perhaps 10 times that amount for roughly the same bandwidth (if not less)?

    ReplyDelete
  19. Maybe the telco's should offer three different plans.

    Strech's plan: Reduced bandwidth without any transfer caps. Then he can download all day and feel he is getting what was advertised to him while paying as little as possible.

    My plan (Average Person Plan): Full bandwidth with reasonable data transfer limits (i.e 250GB+ cap)

    Power user plan: Full bandwidth with no limits on data transfers. This plan should cost substantially more.

    I personally like that the telcos offer high bandwidth downloading even if they dont want customers using this much bandwidth constantly. If you think of their average customer (myself included), they are not going to download 24x7x365. What they are going to do is use the bandwidth in a bursty nature and appreciate the high transfer rate when they do use it.

    If the telcos do starting enforcing caps, then i think they should be required to offer an unrestricted plan even if it is really expensive.

    ReplyDelete
  20. I don't think P2P users are and will be the core of this problem in the future. Let's put P2P users in 2 categories:

    Casual P2P user needs a 1-2Mbit line to download average Xvid movie in real time, and even if he only watches 1/3 of downloaded movies and does that for 4 hours a day, it means he uses no more than 0.5 to 1 Mbps in average, which is some 10% of an average xDSL and less than 5% of an average FTTH connection.

    "Collectors", users that download/upload way more that they can actualy consume (be it to expand their personal software/movie library or simply to gain ranks on torrent community sites) will download as much as possible, but they are limited by their upload bandwidth (because most fast torrent sites force their users to maintain >1 upload/download ratio). They don't care how fast something is downloaded, they just want fast overall connection. Sell them 2M/1M 8am-12am / 20M/20M 12am-8am package for the same price as normal 10M/2M and they will do most of their traffic when other users idle.

    PS: With internet present everywhere and new forms of communications emerging faster than billing models, we are going to see more and more all-you-can-eat offers. Just look at cellular network providers - minute prices are going down, while monthly fees are going up, simply because that's the only way to get their share, when we will do most of all talks via free Wi-fi internet.

    PPS: How much do you pay for a megabyte of traffic, when you send a SMS? Who's getting a free lunch here and who's paying it?

    ReplyDelete
  21. I had to give up the comments after awhile to post something: My father uses some program called Carbonite to back up his data off-site, and suggested I might want to look into the whole thing. I mentioned to him that rumor has it Comcast caps you at about 50GB / month (that they cap at all disturbs me - I'd rather they throttled the bandwidth to something they can support then potentially let me use up my month's allotmet in a couple of days). I mentioned that to back up a 1TB NAS and a couple of servers would take me about 20 months, barring no other traffic and no changes ... not very feasible. Of course, supposedly they compress the data before it's uploaded, and I no doubt have junk accumulated (about 15 years worth of data, all accumulated, probably duplicates everywhere).
    I was present when I made the agreement for cable, and it was not for a usage cap, it was for a minimum bandwidth - what's the point of their recent bandwidth increase if I can't take advantage of it? That's about as useful as saying 'You and the other subscribers on your segment now have double the bandwidth to your house with which you can share the same old bottleneck at the distribution layer'.
    Spare me, and give me something I can use (I pay for unlimited voice / data / text on my phone because I hate surprises).
    How could you possibly subscribe to NetFlix, download games, patches, or even heavily VPN with such restrictions? I'm not interested in a mansion to time-share for 2 weeks / year, I want a modest house I can live in 24x7.

    ReplyDelete
  22. @Mark: thanks for a well-argumented contribution. I absolutely agree with you that a 50GB cap (or, even worse, 5GB cap) does not make sense if you have 10Mbps service ... where your montly cap translates into 11 hours @ full bandwidth.

    ReplyDelete
  23. Okay, now, keeping in mind that 1 mbit/sec costs around 30-40 euros per month all-inclusive. From the customers modem, including (a bit of) tech support, to buying from transit partners ... (and that's heavily negotiated. A new entrant in the market will pay closer to 80).

    So allow me to give you prices for your "reasonable plans" :

    Stretch plan : 50 euros per mbit. We're talking 100 or 150 euros for a decent connection

    Your plan : Since full bandwidth requires oversubscription, a reasonable effort guarantee for good connection will require 2-3 mbit provisioning per mbit that you intend to use. 250 Gb is about 1 mbit. Prices tripled this means about 130 euros/month for some 300 Gb.

    Power user plan : 20 * 40 euros per month = 800 euros per month.

    There you have it. Now all you need to do is to convince just about the entire nation that these are better options than the oversubscribe-and-dump-the-hogs plan.

    These prices, btw, are *not* sufficient to guarantee full bandwidth to either the "my plan" or "power user" type users. That requires oversubscription the other way, and therefore we're talking at least another 20, 30% rise in price per mbit, and it requires that the price of the "my plan" goes up to the power user plan.

    So mr "let's just have different plans" I doubt what you're saying is a viable business model. Customers would run to the competition faster than muslim women run away from their husbands.

    @Ivan Peplnjak

    Thanks for the offer for the "bandwidth sharing" device. We have those actually. We don't use them except for a few users in a few locations (due to their prices).

    @Mark Paquette

    "A modest house" (called a business subscription), is 100% available for us. At prices that can be summarized as "about 50$ per megabit".

    You *can* get a relatively cheap 1 or 2 mbit business connection easily enough. From us, and from comcast too if that's your fancy. But any payable connection will not be capable of backupping 20 tera reliably ... if you want a disk + 20 tera backup for a reasonable price, then you need to look into colocation (put your machines in a central location where you can get 100 mbit for 200-300 euros per month).

    ReplyDelete
  24. Amazing !

    This is the *only* blog i can find that actually understands Why there are limits on how much crap you can up/download on the internet. Mostly it's crap. About 3% is Work kinda stuff.

    The rest of the world, as Consumers, simply expect to be able to consume All they want, with no regard to either physical limits, or financial limits.

    Weird how they behave different in a supermarket.

    I agree that Internet is generaly mis-sold.

    If Internet is to be 100Mb/s all the time for $9.99 a month, regardless of how much you use, then there will be No Internet.

    If i cannot make a few bucks, so i can Eat at least something, then around 10,000 people will no longer have the possibility of an Internet Connection, as i will have to stop making it possible, in order to Eat.

    There really is no Free Lunch for anyone.

    It is as simple as that.

    ReplyDelete
  25. "Otherwise, you’re forcing the majority to pay for the excesses of the few and the majority will eventually revolt (which is why the big socialist experiments didn’t work)."

    That's capitalism, not socialism.

    ReplyDelete
    Replies
    1. I guess you were never living in a hard-core Marx-inspired socialist state ;)

      But yeah, I see your point - the more things change, the more they stay the same :D

      Delete

You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.

Ivan Pepelnjak, CCIE#1354, is the chief technology advisor for NIL Data Communications. He has been designing and implementing large-scale data communications networks as well as teaching and writing books about advanced technologies since 1990. See his full profile, contact him or follow @ioshints on Twitter.