Architectural Agility, or what did DaVinci know that I don't?
/Context.
To understand any technical concept you must first understand the context in which it is applied. When I use the word “architecture” most people will immediately picture a building of some type. I can focus that perception by providing context for the word “architecture”. For example, by saying “Gothic architecture” I can evoke the image of a European cathedral or through the term “Roman architecture” I can give you images of stone columns and porticos. By providing context for the word architecture I can make the meaning clearer to you.
Agile software architecture. Cue sound of a needle scratching across a record.
This is one of the (perhaps many?) places where additional context does not clarify an issue. In most cases, that term evokes more confusion than clarity. What does that term mean? My guess is that most people in the industry will think I am referring to the architecture techniques and tasks undertaken in the context of an Agile software development project.
My meaning – and you knew I had one, right? – is actually referring to the collection of patterns and practices that enable agile business systems. That is, an architectural process that enables agility. We often hear about the concept of “just enough architecture” in the Agile context, but very few understand what that means and even fewer grasp the idea that in order to support that concept you must have a system to back it up.
In medieval times, the builders of the Gothic cathedrals didn’t start their designs from scratch nor did they own have finely detailed plans – the technology didn’t exist to support that and the demand of the benefactors they built for would not allow a “cookie cutter” approach. Master builders moved from city to city with their team of experienced craftsmen bringing their proven architecture solely in their heads. Together they tried out new designs and gradually evolved core patterns such as the flying buttress, which became standard components in cathedrals across Europe. On occasion the architectural approach proved wrong and as a result buildings fell down during construction, causing the builders to re-assess the architecture and try again. This might be one of the earliest examples of Agile methods as they combined architecture and built in what clearly was an empirical delivery approach, but also employing a clear reference architecture and patterns that enabled systematic reuse of proven designs. The analogy falls apart when considering the length of their delivery cycle, though.
There are plenty of real – and current - world examples of how an Agile software architectural system becomes a – if not “the” – primary contributor to business agility. Let’s take one such example: back in 2003 Amazon CEO Jeff Bezos issued the key architectural principles to his development teams that became the basis of their Agile software architecture system (even though, at the time, they were not aware they were creating this system). The edict:
- All teams will expose their data and processes through service interfaces
- All teams will communicate with each other through these interfaces exclusively
- All service interfaces without exception must be designed from the ground up to be externalizable
- The technology used to achieve these goals is up to the individual teams
- There are no exceptions to these rules
While this all sounds dangerously close to implementation, what this did to Amazon was that it forced their design and development staff to create a pattern and process which became a reference for the business. And through that reference architecture Amazon became the agile business it is today.
Before I go too far down the path of deifying Amazon, it is important to note that others have done the same thing with greater and sometimes less success. eBay, for example, adopted the same strategy as did PayPal. All three independently accomplished the same goal: through the application of an Agile architectural methodology, they provided a pattern for success that can be adopted by other organizations. By exposing their core data and processes through consumable and versionable services, they created a new class of application platform, one which responds quickly to change and is technology agnostic.
Although Amazon et al has clearly evolved in pursuit of solutions to specific business opportunities and challenges, it’s also clear they have established a de facto architecture and architecture management system that guides the work of the many product delivery teams and ensures consistency of approach where it’s required. The concept of reference architecture became the enabler of growth and agility through natural technological evolution. The services formed the platform that allowed the extraordinary expansion of all three businesses. The end result is real business agility, and it was delivered by smart architecture backed up by clear policies and realized by agile processes.
A reference architecture articulates primary principles that are typically central to an entire enterprise. These principles are focused on establishing a product and solution independent environment in which agility can be delivered and maintained, with the goal of providing stability to the system over time. This architecture exists purely as a set of models and guidelines and the deployed code and systems are the concrete reflections of that architecture.
Enterprises clearly vary considerably in their make up in terms of geographic and organizational, product and process standardization and differentiation, but typically there will be considerable potential for an inventory of shared assets that leverage agile architecture to support business agility. The assets form the next level of the Agile architectural model, and while still conceptual this level takes architecture closer to the platform on which virtual artifacts may one day be deployed. Platform assets in this architectural level may include:
- Static assets that are designed to deliver common behaviors to all parts of the enterprise. For example, these assets might include core services that establish genuinely enterprise wide delivery of functions like credit card validation or inventory checks. Static assets deliver value by standardizing common business services and processes.
- Configurable assets that are designed to provide common behaviors, but because they are configurable they are engineered to be customizable based on client need. For example, these assets might handle localization (such as tax calculation for various parts of the world) or more complex business processes (such as order fulfillment and product sourcing rules based on the point of origin for an order). Configurable services can provide business value by providing reusable components which implement a common core of business processes and in turn may provide a rapid “time to market” strategy. The trap in this type of asset, however, is in determining whether the business need would best be served with this type rather than an alternate method within a static asset.
- Information architecture and services. Establishing a coherent approach to information is commonly a major issue for large enterprises, and this architecture asset is used to define an integrated approach for structured and unstructured (or what is currently being called “big”) data, traditional extract-transform-load operations, enterprise reporting, regulatory control and the list goes on in regards to how data in interacted with. Information architecture and services provide business value by allowing the business to safely and effectively interact with their information (while maintaining the integrity of that information), and at the same time allows for avenues to be created which allow that information to be leveraged in new and interesting ways.
Diving into the next level of Agile architecture we begin to arrive at a level where we can start to create virtual or code assets. Many architectural models will combine this level with the next one below it, but I prefer to keep them separate for reasons you will see in a moment. The next level of assets we will call the Family architecture, a domain-based framework that provides much more specialized assets particular to a given set of business units or departments such as Customer Order Management, Supply Chain, Risk Assessment and the like.
Near the bottom of the architectural stack is the Product Line architecture. At its heart it is exactly what the name suggest - it’s the architecture for a product offering. The assets at this level have a direct relationship to end customer revenue, and as such usually provide continuity of purpose over multiple releases. Although from a narrow technical perspective the Product and Family architectures might be similar, the way in which a product is managed must mirror the business product life cycle. Family architectures should be designed and engineered for stability, whereas - depending on the business need and customer demand - product line architectures should be designed and delivered with maximum agility and minimum response time.
The final item in the stack is the Solution architecture. As the name implies, it is specific architecture focused on delivering a key component of functionality that stands on the other architectural levels but represents a light and fast implementation of the architecture above it. This level represents the key endpoints of the architectural stack which are subject to rapid change depending on market and business needs. In some organizations, the Product Line level actually bleeds into this level, but from a purity standpoint it probably should not. Of necessity, this level represents the closest delivery point of an asset to what we normally consider “Agile”.
So this is what our architectural stack looks like, from the most conceptual to the final implementation, and from the least to the most Agile-centric deliveries:
| Level | Architecture |
|---|---|
| 1 | Reference |
| 2 | Platform |
| 3 | Common, Configurable and Information |
| 4 | Family |
| 5 | Product line |
| 6 | Solution |
These six architecture levels provide us with a nomenclature for agile architecture that will be central to managing agility into the delivered features. The architecture perspective guides the structure of programs and projects, and the incorporation of architecture and reuse goals into delivery charters for each feature. The stack also provides traceability and governance over realization of core architecture principles.
The question of how Agile Architecture integrates with Agile delivery is likely to prove contentious because by its very nature architecture introduces a direction which seems to contradict basic Agile principles. Yet the lessons to be learned from the Amazon example above are clear. In any large organization customer project demand needs to be managed and aligned with not only the business strategy but also the foundational architecture direction. Senior business management needs to be fully engaged and actively leading the customer community towards feature development under sound architectural direction.
There’s no good reason why the Demand and Definition process can’t adopt Agile concepts, specifically in the areas of cross functional teams, time boxing tasks and architectural backlogs. The outcome of adopting these concepts is visibility and traceability of key strategies and policies that provide real clarity of purpose for projects, which in turn will increase the probability of success for a project. In a typical large enterprise use of existing (or well understood) organizational concepts, adjusted to use aspects of Agile methods as discussed, will meet with less resistance. Here are some examples of how some traditional architectural groups and projects can be adopted for agility:
- The Architecture Review Board could be replaced with a cross functional team consisting of senior representatives from business, product management, architecture and development, with the charter of providing guidance and oversight to all architecture development.
- The traditional role of an Application Architect can be replaced with a design authority team consisting of domain-specific representatives of business, product management, architecture and development. Their mission is to transform the raw customer demand stream into project charters and manage the portfolio view. This team takes responsibility for aggregating and decomposing customer and strategic demand, and chartering Common, Product Line and Family architecture.
- Investigatory architecture projects – short duration projects that validate assumptions prior to chartering composite architecture/delivery projects – should be initiated as part of a definition phase activity concurrent with outline requirements and knowledge discovery surrounding a feature entering the pipeline. This would require using architectural design patterns as a mechanism to increase consistency of architecture decisions and on success, communicate the patterns and lessons learned at sensible level of detail to delivery teams.
This is a recursive model, and the various elements of this approach should be executed at enterprise and program level as required.
You may ask where “Enterprise Architecture” is in this. The answer is that enterprise architecture is a role and responsibility that must coordinate and govern all levels of architecture. Enterprise Architects are most likely to be assigned to a specific architecture perspective level. The notion of, “one architecture to rule them all” really doesn't exist in an Agile world.
Each enterprise should develop its own architecture management approach, and integrate this into an end to end architecture, delivery and governance process. The term “Agile Architecture” should be used to describe and deliver architecture that facilitates the agile business by compliance with reference, platform and other architectures that facilitate evolution, customization, and plug and play. Faster cycle time and quality outcomes are then a function of both the reusable patterns and parts available for assembly during the Agile delivery process.
