Here’s some mandatory reading in case you still believe redundant networking infrastructure cannot fail:
- The network is reliable – a fantastic collection of real-life failures, including all sorts of split-brain scenarios caused by hare-brained schemes to stretch a cluster just a bit too far;
- More stuff on impacts of network partitions from the same author;
- Notes on Distributed Systems for Young Bloods. A must-read for anyone who thinks that ignoring 40 years of hard-learned lessons and controlling a distributed system from a central controller makes perfect sense. Not that it would ever help.