I have a pet theory that it’s all about dependencies.
I work in an area, Network Function Virtualization, NFV for short, that is made up of many complex systems. The most obvious example is OpenStack, which is notoriously considered complicated. To be fair, there are many more high level systems like it that are also part of the ecosystem. (See Kubernetes or almost any SDN.) Complexity reigns.
It’s also “turtle’s all the way down” as OpenStack is built on the Linux ecosystem, which is itself extremely complex, never mind the underlying hardware Linux typically runs on: CPUs, proprietary firmware and drivers, etc. Code is everywhere. And code has dependencies.
My time as Linux systems administrator has given me a particularly strong view of dependencies. Having worked so much with operating system packages (RPMs, deb files) and similar mechanisms (Pip, CPAN, ugh) I’m keenly aware of how important dependencies are, and how challenging they can be to manage.
Unless you write code or manage systems that have packages you might not be as aware just how many dependencies there are. Often organizations pay companies to abstract away dependency issues. Frankly, as far as I’m concerned, a large part of what you are buying when you purchase commercial software is the ability to mitigate the risk of dependencies.
However, ignoring dependencies is a double edged sword. The thesis of this article is that it’s all about dependencies. In my opinion, if an organization is going to be good at managing complex technology they also must be (very) good at managing dependencies. If dependencies exist throughout the technology stack, there comes a point when every organization must deal with them.
I recently read this fascinating paper, “Why and How Java Developers Break APIs“, which studied a set of Java libraries for breaking changes.
…we monitored 400 real world Java libraries and frameworks hosted on GitHub during 116 days. During this period, we detected 282 possible breaking changes
As part of the study, the authors attempted to contact the developers of the libraries to ascertain if the detected changes were actual breaking changes or not. Of those 282 potential breaking changes, 59 (39%) were confirmed by the library developers. The rest remain unconfirmed and are still potential breaking changes.
Thus, over a relatively short period of about 4 months, 59 confirmed breaking changes occurred in only 400 libraries. That is a considerable number.
If I can conclude with anything it’s that in order to manage complex systems (and apparently they are all complex) I believe organizations must be great at understanding and managing dependencies. To deal with complexity requires expertise in dependencies. Certainly we can try to push the risk onto vendors, but at some point we must take responsibility. We own our uptime.