Architecture before Products
Yves Haemmerli, Consulting IT Architect at IBM Switzerland, sent me a thoughtful response to my we need product documentation rant. Hope you’ll enjoy it as much as I did.
Yes, whatever the project is, the real added value of an IT/network architect consultant is definitely his/her ability to create models (sometimes meta-models) of what exists, and what the customer is really looking for.
This modelling phase is crucial. It does not only set a clear methodology and a route map of the project, but it is also the unique way to communicate with the customer, at different levels, what the project is all about and what Architecture Decisions will be necessary. The Architecture Decisions process is itself a methodology to put in place at the beginning of the project. In large SDN projects for instance, I use my models for instance to establish a detailed Work Breakdown Structure (WBS) for the project
During this modelling period, it is highly recommended to keep ourselves (as far as we can...) outside pure product oriented considerations. In principle, modelling should to be product independent. Once the first drafts of the target architecture (models) are established, then, and only then, product presentations are necessary. Here too, I agree with you: first take the time in advance to have a close look on vendors’ product documentation (says a lot about the vendor...), understand the key concepts, and be prepared for a product presentation meetings to focus the discussions on implementation architecture, scaling, limitations, dependencies, etc.
Yves is an Electronics Research & Development engineer. He specialized in Digital Signal Processing and data encoding/decoding in the early years (1982-1990) of digital audio at Studer-ReVox, before joining the multi-protocol networking community at IBM in 1990 (in his own words, approximately at the same paleolithic period when dinosaurs disappeared ;-)
At the beginning we do select some products using manuals but almost always it did not work in our case. Sometimes because the vendor statements about performance require very specific test conditions (trivial examples: MTU size was much bigger than we used, signing algorithm was supported by HW vs what we needed which was handled by SW).
Then we talk to the vendor to help us select the right hardware. This often helps us but in some cases it also fails. This time we fail because of the system dynamic / timing.
Some vendor hardware (general purpose hardware for the price we are willing to accept) is too slow to setup for example new FW session (I selected trivial example for simplicity; actually we have more subtle issues like cached packet reordering, etc).
So we do more test upfront in the lab to identify most critical parts in the vendor product and we talk to the vendor again.
This part is critical because the vendor has to decide if he can fix the issue or enhance d the product within the current architecture, or we need to buy something more expensive.
Critical because we risk the time spent on integration of the potentially working product before we are 100% sure that the potential fixes / enhancement will solve the performance problem on the selected architectures.
It happens that vendor is wrong and cannot meet the expectation within the architecture and we need to repeat the process several times.
I know our requirements are somewhat challenging because the product is specific (Mission Critical Voice Communication Systems).
I want to say that for quite complex systems the manual might not be enough. The crucial part is the vendor involvement and willingness (and risk associated with this) because some especially timing critical requirements cannot be verified without heavy testing before product selection. And many times the vendor has to add new feature or tune performance of its product.
What I wanted to point out was that having decent documentation is a mandatory prerequisite. If the $vendor can't get that act together, who knows what will happen when you hit the next roadblock.
Funny story: When I reported an issue of packets reordering inside a box (two different vendors having parallel processing inside the box), both vendors answered immediately: it's impossible!. Yes, it was impossible according to the manual (and the officially available architecture details)!
But both vendors fixed the issue because the issue was real!. It is all about the timing (time spent on FW session setup vs packet pace, talking about UDP of course).
This case shows how the manual is only very first step, and how the architecture might mislead even the vendor.