Does your team have a “build server guy”, while everyone else stays away? Or worse still a separate “build server team”? I’ve certainly worked with a few development teams where I’ve seen these organizational anti-patterns – which at best can be inefficient, and at worst can be extremely disruptive. One of the challenges when introducing build scripting and other “devops” things can be that many developers either aren’t interested, or see it as a separate set of skills that they don’t know and don’t value learning. These skills are of course a specialism, just like there is normally one member of a team who is the specialist in user interface design or who specializes in automation testing. There can be a great temptation especially in organizations who are less familiar with cross-functional autonomous team to create specialist roles – which creates interesting consequences which will be familiar to those who understand Conway’s Law.
Coupling is often talked about in the context of fine detailed interactions between software components, but as a concept also applies to many other things. Quite simply, two things are coupled if a change in one can affect the other, or if one depends on the other. Excessive coupling is highly undesirable, because it leads to brittleness and a tangled web of dependencies that is hard to reason about, but some coupling is essential. Just as components can have dependencies on each other, so can people and teams, and this could also be termed as coupling. In one of the worst example I’ve seen, a software delivery team were held by organizational structure at arms length from being able to implement their own build and deployment pipelines, which was instead the responsibility of a separate team. Clearly, the delivery team needed to have a working build pipeline in order to release software, and so they were coupled directly to the artificially separate “devops” team. To make matters worse, this team had their own prioritization and backlog, their own objectives and agenda, and were distant from the main team. The consequences are predictable and dire – the team would frequently be blocked in delivering because of misaligned schedules, and there was a massive overhead in trying to get anything done. Another consequence of this was that the solutions that the separate team were often not fit for purpose and did not meet the needs of the delivery team, and because they weren’t themselves dependent on the result, there was very little empathy or urgency to improve.
It’s ok and sometimes necessary for some things to be coupled, so long as that coupling is understood and managed effectively. It’s my experience that build and deployment scripts often need to change alongside the codebase. Some changes – such as the introduction of new tooling, or changes to the deployment topology – will only be built and deployed successfully by a corresponding script change. The easiest way to achieve this is simply to leverage existing version control techniques – the scripts should be in the same version control system as the things they relate to. I often see separate repositories for scripts, which can work if the scripts are packaged in a versioned way, and brought in as a dependency to the codebase – which allows them to be reused – but in general my preference is for the scripts and codebase to be in the same repository. I often apply a consistent pattern to the root level folder structure, with a “Source” and “Tools” folder or similar, to contain code and scripts respectively. There should almost never be a need to change the build server itself, because all it does is call into the scripts in the repository.
I like to encourage my teams to understand and write their own scripts, for a number of reasons – the most obvious of which is to discourage the key-man dependency or single-point-of-failure from having one person solely own this responsibility. It’s important to treat build scripts the same as any other code, trying to keep them maintainable and not allowing them to become overly complex or opaque, which can easily happen if there is only one person with a relatively high level of comfort in scripting. Ideally the team should embrace a definition of done for stories that goes all the way through to including these script changes, so that a feature is only considered ready if it can be deployed in a repeatable and consistent way. It makes no sense for an engineer to consider a feature “done” but they are waiting for someone to write build or deployment scripts – the same way as a feature that hasn’t been tested can never be called “done”.