Speculation: This is how I would build QFabric

Three months after the QFabric launch, the details remain shrouded in mystical clouds, so let’s try to speculate what they could be hiding. We have two well-known facts:

  • QFabric has three components: QF/Node (edge device), QF/Interconnect (high-speed core device) and QF/Director (the brains).
  • Juniper is strong in the Service Provider technologies, including MPLS, MPLS/VPN, VPLS and BGP. It’s also touting its BGP MPLS-based MAC VPN technology (too long to write more than once, let’s call it BMMV).

I am positive Juniper would never try to build a monster single-brain fabric with Borg or Big Brother architecture as they simply don’t scale (as the OpenFlow crowd will learn in a few years).

They would likely keep the individual components in the QFabric pretty autonomous and use distributed processing while using QF/Director as the central management/configuration device (similar to UCS manager in Cisco UCS).

Looking through my rosy MPLS-tinted glasses, the layer-2 QFabric could look like this:

QF/Interconnect is a P-router that does super-high-speed MPLS switching augmented with something like ETS (for differentiated queuing) and PFC (to make the fabric lossless). The number of MPLS labels is reasonably low, so you need very small TCAMs.

Also, don’t forget that Juniper loves P2MP pseudowires and LSPs, so it’s only natural to use those to flood between edge devices.

QF/Edge is a PE-router running BMMV. VLAN handling is part of BMMV, as is multi-chassis link aggregation and active/active multihoming. The final touch: make sure the QF/Edge devices always become STP roots (the whole QFabric would have to appear as a single STP device to the outside world) and you’ve solved the STP integration.

QF/Director is MP-BGP route reflector and the central configuration/management device.

As with all speculations, this one is probably totally wrong – Juniper claims its QFabric is revolutionary and there’s nothing too exciting in the architecture I’ve just described (apart from the fact that it might actually work and scale) ... so I’m still as clueless as most of you are and looking forward to getting more in-depth QFabric details (everything I’ve seen so far is pure marketing fluff).

19 comments:

  1. Ivan

    I've been trying to get Juniper on the Packet Pushers to talk about QFabric na d give us some hints on how it works - but their PR team appears to be scared of Social Media. So far, it's been real hard work to get an intelligible response and I'm close to giving up.

    Here's hoping.


    greg

    ReplyDelete
  2. czesc Ivan , well as you know the discussions are still under NDA regarding 'stratus' so limited technical details can be shared today.
    I think the two interesting ietf drafts related somehow to "stratus" can already be discussed:
    http://tools.ietf.org/html/draft-raggarwa-sajassi-l2vpn-evpn-02 and the "Data Center Mobility based on BGP/MPLS" http://www.ietf.org/id/draft-raggarwa-data-center-mobility-00.txt.
    The layer 2 IS-IS ( rfc6165 ) is also 'supposed' to be there but used for next-hop switch-id(sw-mac) reachability and not end host MAC@ advertisement. The interesting thing is that the solution is not following all the current Trill or SPB like trends , neither we don't see any juniper authors in these drafts.
    The whole c-plane stuff is supposed to be hidden from the user point of view as it is with the currently available virtual-chassis of the EX series (EX8200 with XRE or EX4200). More discussion will certainly take place in a couple of weeks :-D at the public level.
    Cheers ,
    michal

    ReplyDelete
  3. czesc Ivan , well as you know the discussions are still under NDA regarding 'stratus' so limited technical details can be shared today.
    I think the two interesting ietf drafts related somehow to "stratus" can already be discussed:
    http://tools.ietf.org/html/draft-raggarwa-sajassi-l2vpn-evpn-02 and the "Data Center Mobility based on BGP/MPLS" http://www.ietf.org/id/draft-raggarwa-data-center-mobility-00.txt.
    The layer 2 IS-IS ( rfc6165 ) is also 'supposed' to be there but used for next-hop switch-id(sw-mac) reachability and not end host MAC@ advertisement. The interesting thing is that the solution is not following all the current Trill or SPB like trends , neither we don't see any juniper authors in these drafts.
    The whole c-plane stuff is supposed to be hidden from the user point of view as it is with the currently available virtual-chassis of the EX series (EX8200 with XRE or EX4200). More discussion will certainly take place in a couple of weeks :-D at the public level.
    Cheers ,
    michal

    ReplyDelete
  4. Ivan Pepelnjak01 June, 2011 14:47

    Thanks for the pointers to IETF drafts. Both of them rely on at least two technologies I've mentioned ... not bad ;)

    ReplyDelete
  5. michal
    czesc Ivan , well as you know the discussions are still under NDA regarding 'stratus' so limited technical details can be shared today.
    I think the two interesting ietf drafts related somehow to "stratus" can already be discussed:
    http://tools.ietf.org/html/draft-raggarwa-sajassi-l2vpn-evpn-02 and the "Data Center Mobility based on BGP/MPLS" http://www.ietf.org/id/draft-raggarwa-data-center-mobility-00.txt
    The layer 2 IS-IS ( rfc6165 ) is also 'supposed' to be there but used for next-hop switch-id(sw-mac) reachability and not end host MAC@ advertisement. The interesting thing is that the solution is not following all the current Trill or SPB like trends , neither we don't see any juniper authors in these drafts.
    The whole c-plane stuff is supposed to be hidden from the user point of view as it is with the currently available virtual-chassis of the EX series (EX8200 with XRE or EX4200). More discussion will certainly take place in a couple of weeks at the public level.
    Cheers ,
    micha

    ReplyDelete
  6. czesc Ivan , well as you know the discussions are still under NDA regarding 'stratus' so limited technical details can be shared today.
    I think the two interesting ietf drafts related somehow to "stratus" can already be discussed:
    http://tools.ietf.org/html/draft-raggarwa-sajassi-l2vpn-evpn-02 and the "Data Center Mobility based on BGP/MPLS" http://www.ietf.org/id/draft-raggarwa-data-center-mobility-00.txt.
    The layer 2 IS-IS ( rfc6165 ) is also 'supposed' to be there but used for next-hop switch-id(sw-mac) reachability and not end host MAC@ advertisement. The interesting thing is that the solution is not following all the current Trill or SPB like trends , neither we don't see any juniper authors in these drafts.
    The whole c-plane stuff is supposed to be hidden from the user point of view as it is with the currently available virtual-chassis of the EX series (EX8200 with XRE or EX4200). More discussion will certainly take place in a couple of weeks at the public level.
    Cheers ,

    michal

    ReplyDelete
  7. Ivan Pepelnjak01 June, 2011 15:23

    Thanks for the pointers to IETF drafts. Both of them rely on at least two technologies I've mentioned ... not bad ;)

    ReplyDelete
  8. Funny, because I had no problem getting Juniper to talk to me...

    ReplyDelete
  9. Greg

    Juniper started marketing almost 2 year ago and after long awaited queue they entered in data center and now they donot want to reveal about the hidden facts O:-) of Qfabric .

    regards
    Shivlu Jain

    ReplyDelete
  10. No offense, but Greg doesn't have pics of Juniper equipment on his home page, has at least 5X your following and one the most popular networking blogs and podcasts on the internet.

    ReplyDelete
  11. Abner Germanow03 June, 2011 00:55

    Hi Ivan,

    I commented on Greg's post. The comments apply to your post as well:
    http://etherealmind.com/juniper-qfabric-my-speculation-too/#comment-10974

    Thanks for spending the time on this.

    Cheers,
    Abner

    ReplyDelete
  12. Well...you could do a little google search and find this ;-)

    http://www.google.com/url?sa=t&source=web&cd=1&sqi=2&ved=0CBYQFjAA&url=http%3A%2F%2Fno.convergencepoint.westcon.com%2Fdocuments%3Bjsessionid%3D212DB5A0340743BB9C9883D2FF15815A%3FdocumentId%3D40653%26filename%3Djuniper_qfx3500_sales_preso.pdf&rct=j&q=juniper%203500%20sales&ei=vcbpTa-VC4T0swPKrJjbDQ&usg=AFQjCNGks8le_Udl_OeZxuQoSPu1VaDpJg

    Couple things have jumped out for me

    1.) For all their talk of how much better switching is than routing its interesting that L3 features are not available for a few quarters
    2.) They seem to do a rather poor job of competitive research as their details on Arista, Cisco and Brocade are off in a few places
    3.) For all the Qfabric excitement the 3500 is listed as only Qfabric Ready*
    4.) Latency seems a bit high compared to others
    5.) 48 10G ports + 4 40Gs or +16 more 10G ports on one ASIC is VERY impressive
    6.) Until the 40G support is ready, scaling to 6000 ports comes out to 96ish switches and if you really need to link every switch to every switch I don't see where you plug in the servers (in fairness to Juniper, they do ISP stuff, maybe they didn't realize servers were needed)
    7.) I keep hearing mixed messages on their use of BGP and IS-IS. I am now starting to wonder if they use ISISL2 to keep with SPB and then a separate additional control plane in BGB.

    ReplyDelete
  13. I think ccie25655 was making that same point. If you _don't_ have "5X your followers and one of the most popular networking blogs and podcasts on the internet" it's probably quite easy to get info.

    However you rubbing that in wasn't very nice ;)

    ReplyDelete
  14. Ivan Pepelnjak04 June, 2011 08:43

    Nice find, thanks for sharing!

    Lots of useful information, but nothing at all on QFabric

    ReplyDelete
  15. Fair enough. Apologies @ccie25655 :)

    ReplyDelete
  16. Just that it's Qfabric Ready =)

    I think I will declare myself "Independently Wealthy Ready*" and see what happens

    ReplyDelete
  17. Abner Germanow06 June, 2011 06:18

    Hi Jon, Nice to see Brocade joining the conversation.

    1) L3 features for the QFX 3500 standalone switch will be part of JUNOS 11.3, if you were a customer in need of L3 today, we might be able to help you out, although the hedge funds and traders buying the QFX3500 today are mostly using it as a L2 switch. Incidentally, the L3 functions don't create a performance hit on the QFX 3500 vs L2.

    2) Please point out where this is, people are constantly shipping new products in this market. We've seen a few compare 24 port to 48 port switches incorrectly as well.

    3) Which was something we stated very clearly at the launch and to every customer, analyst, and press person for the last 6 months.

    4) Compared to some of the 24 port switches, perhaps, but the standard deviation as the switch loads up is rock solid as has been validated as part of the STAC tests.

    5) Thanks :-)

    6) We don't expect anyone to build a 6,000 port network without the other components of the fabric. As you point out building a fabric that does more than create an any to any connectivity out of just Ethernet switches is doesn't scale.

    7) Does anyone run BGP across the backplane of their switch? No they do not.

    Cheers,
    Abner
    Juniper Networks

    ReplyDelete
  18. Nice to see Brocade joining the conversation.

    >1) L3 features for the QFX 3500 standalone switch will be part of JUNOS 11.3, if you were

    Oh! Egads no! I'm by no means here as a "Brocade Representative" nor am here as an "Ex Juniper Representative" or as a "CULT of the people who want to put their Brains into Robots" for that matter. Just a geek who doesn't sleep much.

    >a customer in need of L3 today, we might be able to help you out, although the hedge >funds and traders buying the QFX3500 today are mostly using it as a L2 switch. >Incidentally, the L3 functions don't create a performance hit on the QFX 3500 vs L2.

    No no, I'm very happy with no L3 at this level. I was just surprised as Juniper seems to in general favor routing over switching. I think for many applications a nice flat L2 network is much better.

    >2) Please point out where this is, people are constantly shipping new products in this >market. We've seen a few compare 24 port to 48 port switches incorrectly as well.

    If you follow the above link to the sales deck publicly available, I was referring to the bandwidth, latency, and power numbers you had for Arista, Brocade etc. Some were off.

    >3) Which was something we stated very clearly at the launch and to every customer, >analyst, and press person for the last 6 months.

    Oh! My bad then, my apologies. I was incorrectly told that the 3500 was the first Qfabric switch.

    >4) Compared to some of the 24 port switches, perhaps, but the standard deviation as the >switch loads up is rock solid as has been validated as part of the STAC tests.

    I love the concept of rock solid statistics based on standard deviation :) However you are right, compared to other 48 port switches 900ns is quite respectable. Would be nice to have something in the sub 500 or 600ns range, easier to compete with IB. We should all get down to around 260ns soon I hope.

    >5) Thanks

    Juniper has awesome ASIC groups

    >6) We don't expect anyone to build a 6,000 port network without the other components >of the fabric. As you point out building a fabric that does more than create an any to any >connectivity out of just Ethernet switches is doesn't scale.

    AH!! that makes much more sense.

    >7) Does anyone run BGP across the backplane of their switch? No they do not.

    Wow..umm...sorry? Did I say something wrong? Well I must admit I don't know of anyone running a BGP derived control plane. Doesn't seem like a horrible idea to me. However maybe you are right, maybe no one ever will.

    Maybe you can clear it up?

    I thought Qfabric used SPB. However I have had multiple people claim QFabric will use some BGP variant. Which confused me.

    So is Qfabric SPB? I mean I know there is secret sauce and all that, but you are saying _no_ BGP is used in Qfabric? So then it's SPB with IS-IS then?

    I apologize if I offended you Mr. Germanow. I'm not very good with humans.

    ReplyDelete
  19. Abner Germanow13 June, 2011 08:48

    No worries Jon. All good conversation. :) Apologies for turning you into the voice of your employer, but then again thanks for signing on under your own name. We suffer from a fair amount of astroturfing in the networking industry that doesn't help anyone.

    Cheers,
    Abner

    Juniper Networks

    ReplyDelete

You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.

Ivan Pepelnjak, CCIE#1354, is the chief technology advisor for NIL Data Communications. He has been designing and implementing large-scale data communications networks as well as teaching and writing books about advanced technologies since 1990. See his full profile, contact him or follow @ioshints on Twitter.