Its not just a good idea, its the law

For rather a long time human interaction has been governed by the rule of law.  Defining the principle, we accept that law is generally regarded as regulations or rules established within a community by the authority of that community.  These rules are accepted and enforced by the community or by an authority that the community appoints to enforce them.  That was a long way to go to define what a law is.  There is a point to the definition, though, and it is this: outside of community, what we declare to be “law” are in fact more like guiding principles. 

Take software development, for example.  When talking about the “laws of development” there is no way to apply the concept of absolutes such as those implied by the definition of “law” to something which by its very nature does not traffic in absolutes.  Put another way, trying to declare a principle of development as a law (and therefore subject to enforcement and penalty when not adhered to) subjects you to serious jail time as you spin up all the myriad ways in which an exception for the rule can be declared.

That being said, most development efforts accept certain rules as a law of sorts, and so development can be guided – or at the very least, predicted – based on how the laws are applied and enforced. 

The company I work for has had some churn recently regarding the Agile development process, and it seems as if there is significant confusion regarding how to implement a scrum methodology that works.  Along with that confusion comes a lack of understanding what the scrum process is, and so as a result I went back to a book I had previously read to get some guidance from a couple of guys who know their way around the process.  The book is called Exploring Scrum: The Fundamentals and it was written by Doug Shimp and Dan Rawsthorne.

Back to the topic at hand – the laws of development.  The intro to the book was written by Jeff Sutherland, who along with Ken Schwaber are acknowledged as the fathers of scrum.  In the intro, Jeff talks about some of the accepted laws within the scrum process, specifically Conway, Humphrey and Ziv’s laws.  And while reading the intro, I began to understand a little of how software projects can jump off the rails.  Well, I can see how our trouble can be resolved by accepting and adhering to one of these laws.

Let’s talk a little about the other two laws first.  Humphrey’s law is an odd duck – or, perhaps, centipede.  The law comes from George Humphrey, a psychologist who developed the principle in a 1923 book named The Story of Man’s Mind.  He illustrates his law through a poem about a centipede who moved just fine until he was asked how he was able to move so many legs without thought.  As the centipede considered the question, he fell down as he no longer knew just how to make the legs move properly.

Regarding software development, the law is defined this way: a customer never really knows what it is that they want until they see working software.  The more effort put into considering how the software works has the inverse effect of actually slowing down the development process.  For a long time, this meant that the customer did not see working software until the end of the development cycle which in turn makes it too late to accommodate changes they may want to implement their vision. 

Within the Agile world, this now means that we strive to deliver early a prototype that allows the customer to see how the software (or, more appropriately, a portion of the software) should work and allow development to handle the inevitable changes without disrupting the delivery.  As a customer interacts with the prototype they become more comfortable with the system potential and can direct the development process.  This iterative review and recommendation process leads to fewer surprises, better architected solutions and much happier customers (and developers) because it is a collaborative endeavor.  No one “forgets” how to do their job because they never reach a state of paralysis through overthinking about the goal.

Along similar lines, Ziv’s law states that development is inherently unpredictable because very few (if any) people associated with a given project fully understands the problem, let alone how to approach a solution.  When creating something new, it is impossible to fully account for all of the functional and non-functional requirements regardless of the similarity the project presents to predecessors.  Certainly forecasts can be created from those similarities with varying degrees of accuracy, but in the end it is the unknown – the “not knowing what you don’t know” – which derails projects.

Again, Agile looks to defuse a part of this uncertainty through a shorter and more collaborative iteration process.  By breaking the problem down into smaller units of work, it is possible to engage subject matter experts for each phase to help reduce the amount of “unknown”.  You can’t eliminate it, but you can reduce the risk.  If you don’t attack the risk, the risk will attack you. 

So far, the first two laws show us that software development is unpredictable and that you can’t be sure of what you are developing until you actually develop it.  The final law Sutherland talks about is the one which hit a resonant chord for me.  It is Conway’s law, and it can be summed up this way: the structure of the group writing code will be embedded within the code.  In the end, this may be the reason why we are having difficulty in implementing a viable scrum strategy.

What does that actually mean?  Well, a traditional hierarchical structure within a development team has a tendency adversely affect the code the team puts out.  Put another way, when there is a standard pyramid structure to a development organization, there is a desire to develop code in the same fashion, which leads to poor architectural decisions, bad object choices, unmaintainable code and in general a brittle application.  If there are communication problems between teams within an organization, the code those teams develop tends to communicate poorly with its peers. 

There are a lot of people who assume Conway’s law is some sort of commentary on dysfunctional development teams, but I don’t think this was Melvin Conway’s intent - or, at least, perhaps his primary intent.  Since he first stated the law in 1968 there have been numerous studies which present overwhelming evidence that this is in fact the case.  Conway states on his web site:

(Conway's law) was not intended as a joke or a Zen koan, but as a valid sociological observation.  It is a consequence of the fact that two software modules A and B cannot interface correctly with each other unless the designer and implementer of A communicates with the designer and implementer of B.  Thus the interface structure of a software system necessarily will show a congruence with the social structure of the organization that produced it.

By nature Conway’s law is passive.  It is simply a meditation on how software tends to follow the social order of the developers.  It is only when you consider the law in the light of your own organization that you can begin to turn the law into an active agent for change.  And that is the heart of my lesson.  Yeah, it took a long time to get there, but I get paid by the word, so sue me. 

The moral of the story?  If you want to create a functional software development team, implement a process that can help overcome the first two laws and create code which is both resilient and effective then you have to take the first step of assessing your team and organization.  That means taking a long, hard look at how (and why) you do things and be ready to change to achieve your goals.  More often than not, companies assume that process trumps tradition, so they look to implement rules over a structure which won’t support the rules or which yield wonky results.  Restating Conway’s law into an active rule, we should create teams, communication structures and operational processes to match the architecture we want to develop.

Attempting to apply a process like scrum over an organizational structure which won’t support it is much like putting a Band-Aid on a severed limb.  And I will leave you with that visual.