Stop Using Agile for Infrastructure

AI Generated Image

Agile methodologies (especially sprint-oriented) are the wrong approach for delivering software infrastructure. Typical implementations introduce inefficiencies in the name of an unnecessary prospect of adaptation to rapid change. Moreover, they favor small, cross-functional teams that are expected to be highly autonomous and sized & organized in reflection of the modular functionality we expect from modern, cloud-native architectures. But software infrastructure is not elusive like consumer interest; it is the factory itself, not the manufactured items[1].

Don’t get me wrong, Agile is great. It’s almost never actually implemented as per the manifesto in my experience but even when bastardized (in particular, #1 goes bye-bye very quickly) it has a place in software delivery as an ideal. More tactically relevant, Agile frameworks can provide properly composed teams with a pattern for success for specific use cases. This essay makes little distinction between Agile the ideal and Agile frameworks; neither is relevant. It’s simply the wrong approach.

If you have external end-users and want the room to evolve further engagement/requests/etc. then there are plenty of excellent patterns. They give you the freedom to create modular, cross-functional, self-organizing dev teams that can work at their own velocities and evolve as necessary.

This is not how we should deliver Infrastructure as Code (IaC). Our end users are internal and they are not fickle or coy. They are security teams and compliance teams. They are our dev teams and we’ve assured them that they have a safe space to work within to build their software delivery life cycle (SDLC) tooling that enables their Agile patterns. Most importantly, our product end state is fully determinable (whether it’s fully determined is one thing, but it is determinable, with proper effort, knowledge, and wisdom).

While decoupled code is still the ideal, and certainly teams can (and should) be sized and segmented (e.g. by domain) they need not (nor should not) be siloed. They are working toward a common product rather than building independent components that can be reconfigured and reused ad infinitum. The team determining organizational/account-level policies must work in lockstep with the team managing networking components. There is no chance that (tomorrow or next sprint) your security team is going to decide based on “market research” or “focus groups” that you don’t need an IDS or SIEM. There is no chance your networking team is going to change the CIDR schemas of your network topology overnight without a damn good reason and probably multiple levels/rounds of approvals.

Another major drawback to Agile IaC is the expectation to scale the dev team. In reality, at least today, there are usually too many dependencies between layers of IaC to create too many degrees of separation between teams[2]. Simultaneously, it is too costly to have any team duplicate another team’s work. Ideally, code is curated by a highly skilled team that provides code oversight and governance across all IaC teams (perhaps it is an independent team; perhaps it is a committee drawn from across teams). Treat this team as an internal Open Source Software (OSS) dev team that provides consistent, safe, stable code to operators, who will in turn purely provide configuration details consistent with the standard operation of the latest published code[3].

Perhaps the most obvious drawback is the inclusion of rituals and cadence meetings that are widely superfluous if enough effort was put into the design, upfront. Standups may be worth keeping, but especially regular planning and retrospective meetings should be avoided. Better use of collective time would be put into proper code reviews to ensure adherence to standards and specifications and to expand accountability.

The last, and perhaps most rankling drawback to Agile IaC is the expectation of a Minimum Viable Product (MVP). The minimum viable infrastructure product for a serious digital organization must be secure from external or internal attacks, must be observable and fully operable (especially to provide emergency shutdown), and must provide all the components for a software delivery team to produce its MVP. Do not (do not) call “working” IaC “done”. The minimum viable infrastructure product is a complete infrastructure product!

You can probably get to a minimum complete infrastructure before your IaC’s final state. It can also be designed (with care) to enjoy a high degree of modularity. Over time, your infrastructure will change: components will be replaced and upgraded as new technologies and techniques emerge. But don’t confuse the motivations behind replacing your IDS with the impetus driving Agile; these changes should be fully planned and scheduled well before development begins.

It is clearly not impossible to deliver software infrastructure with an Agile framework; plenty of organizations do it. However, it may be a bit like steering a speed boat having only ever before driven a car: you may not crash, but there will be course corrections. The most effective strategies will be highly deliberate and anticipatory. They will have studied the course before they ever start the engine. From start to finish, the path is clear and predictable. There is no need for an approach that anticipates change that does not exist.

---

Footnotes:

[1] As such it is often full of dependencies, and while they need not be brittle, they should not be compartmentalized to the point of being mutually oblivious.

[2] It may be theoretically possible to organize such a segmented team in some fashion that adheres to enough of the Agile ideals to be considered an Agile framework, but this would be extremely contrived (and rife with dependencies) to the point where it is effectively indistinguishable from Waterfall.

[3] Kanban is actually a very reasonable way to run an IaC project, but it does suggest a velocity that roughly grows linearly with team size. Again, this is rarely possible, in reality, with Infrastructure as Code due to strict dependencies between tasks. One would have to be Orwellianly vigilant to express and enforce these dependencies perfectly; even still, you risk bottlenecks and single points of failure.

Infrastructure As Code

This article was originally published on Medium on 08/29/2022