One of my readers sent me a container security question after reading the Application Container Security Guide from NIST:
We are considering segregating dev/test/prod environments with bare-metal hardware. I did not find something in the standard concerning this. What should a financial institution do in your opinion?
I am no security expert and know just enough about containers to be dangerous, but there’s a rule that usually works well: use common sense and identify similar scenarios that have already been solved.
Do you use the same vSphere cluster for dev/test and production environments? If you do, you might consider using the same physical servers for dev/test/prod container environments (but keep going). If not, there’s your answer.
Next, do keep in mind that the containers don’t provide nearly the same level of isolation as hypervisors, as all the containers use the same underlying kernel (unless you’re using one-container-per-VM approach). For more details, watch the Server Virtualization section of Data Center 3.0 for Networking Engineers course.
Conservative security specialists keep pointing out that the number of known Linux kernel exploits far exceeds the number of known hypervisor exploits, and thus recommend using separate Docker Swarms (or Kubernetes clusters) for different environments, or even for different application stack.
Whether you run these clusters on bare metal servers or on virtual machines is another question and depends on the flexibility you need, and the resources required by each cluster.
Matthias Luft covered many of these topics (including relative insecurity of containers) in his ipSpace.net webinars.