Harry Taluja asked an interesting question in his comment to one of my virtualization blog posts:

If vShield API is no longer supported, how does a small install (6-8 ESXi hosts) take care of east/west IPS without investing in NSX?

Short answer: It depends, but it probably won’t be cheap ;) Now for the details…

Why would you want to do that? Always start with why do you need this, not how will we do it? Starting with the root cause might show you alternate approaches that give you the results you need without facing the problem you thought you had.

In this particular case, you might want to consider locking down the security zones with ACLs that are available in vSphere 6.

Like with all other good vSphere networking stuff, you get ACLs only in the distributed virtual switch, which comes with the reassuringly expensive Enterprise Plus edition. For more details, watch the vSphere 6 networking deep dive webinar. Oh, and even the free version of Cisco Nexus 1000v won't save the day - it does have ACLs, but runs only on top of distributed virtual switch.

What exactly do you want to do? Harry mentioned IPS, which requires inline appliance to be reliable (sending TCP RST packets from an out-of-band appliance might be already too late and definitely won’t stop the next Slammer).

If you’re OK with having an IDS, not an IPS, you don’t need an inline solution but an out-of-band one that has to work along the lines of using SPAN ports on physical switches.

Can you afford it? How much east-west traffic do you have? How much will the license (or the appliance) you need to process that traffic cost (unless you use Bro IDS, of course)? How many CPU cores does the IDS/IPS software you use burn to inspect 1 Gbps of traffic? Does it make sense? How will the extra latency caused by all traffic passing an inline appliance affect your application performance? What happens when the central chokepoint (your IPS appliance) becomes overloaded?

HP had an interesting idea: redirect all the east-west traffic to their TippingPoint appliance. Needless to say, the multi-gigabit versions of their appliance were ridiculously expensive.

How would you do it?

You probably realized why I made this question the last one in the blog post ;) Go into the weeds of the technical details only after you thoroughly understand the business problem.

If you think you need an IPS solution, you need service insertion capabilities. vPath technology in Cisco Nexus 1000V does that, but I haven’t found a single IDS/IPS solution that would work with vPath.

You might also consider using Nuage VSP or Juniper Contrail, as they both have some service insertion capabilities. I don’t know how expensive Nuage product is, but even if you go with open-source Contrail, considering the amount of work you’d have to invest to get some of these ideas up and running, buying NSX might be a better deal.

Want to know more about service insertion? Watch the SDN Use Cases webinar.

If you’re OK with an IDS solution, your life becomes much easier. All you need is traffic mirroring capabilities, which you can get in one of two ways:

• Deploy an IDS appliance on every ESXi host, connect it to the virtual switch with a VLAN trunk, and allow the VM NIC to go into promiscuous mode. Oh, your IDS doesn’t support VLAN trunking? Yell at the vendor, and start using Bro IDS.
• Deploy one or more centralized IDS appliances and send the traffic to them using SPAN capabilities built into vSphere using either a monitoring VLAN or RSPAN (traffic encapsulated in IP tunnel). Oh, your IDS cannot decode encapsulated traffic? See previous bullet.

More details? You know you’ll find them in the vSphere 6 networking deep dive webinar.