Two days ago I posted Non-Functional Requirements for the Software Architect in which I suggested that quality attributes or non-functional requirements could be categorized into a two by two grid, operational and non-operational by internal and external.

Thinking further along those lines led me to picture the illustration below and hop out of bed to put it to virtual paper.

Most quality attributes will fit into one or two of the big four: security, performance, usability and modifiability. And each of these largely corresponds to its depicted quadrant.

Security is largely an internal concern. Often external stakeholders will profess their concern with this attribute, and hence the Venn-like spill over into the external quadrant. Generally the concern for security and assuring security falls within the domain of internal stakeholders and the implementation team.

External stakeholders are mostly concerned with performance and usability. There are a number of other quality attributes such as availability that fall within or at least are closely related to one or both of those.

Modifiability is almost exclusively a concern of internal stakeholders and implementation teams. Interoperability and testability are largely related to modifiability while interoperability nearly always shares at least some security concerns.

You may mix these differently to match your circumstances and priorities, but the illustration covers the majority of software development efforts but of course does not spell out the details. For guidance on that, I recommend my previous post. If you are concerned with software architecture, perhaps this will help you to visualize how these quality attributes or non-functional requirements compete and compliment one another.

To paraphrase Bass in Software Architecture in Practice, an architecture will either inhibit or enable the achievement of a system’s desired quality attributes. Understanding your desired quality attributes well will drive the critical decisions in your architecture. And your architecture will then provide the requisite containers into which you will place your functionality. To quote, “Functionality is not so much a driver for the architecture as it is a consequence of it.”

While I am not completely certain that is always the case, it is a principle that is well worth considering when creating and refactoring the architectural elements of your system. And it is my hope that by seeing this illustrated, it will get you thinking about how your own desired quality attributes interact with internal and external teams, which are operational and non-operational, and how they may compete with or compliment on another.

I grew up on a small farm in eastern Utah a few miles west of Roosevelt. My dad is still farming but on a smaller scale these days. He and my mom dropped by for a visit the other day. They asked about work and as I described to them what I do, it struck me that there are many similarities between agile software development and farming.

Here are a few that came to mind:

There is one very big difference between software and farming. The former pays better and is easier.

Countless failed software development projects have been kicked off with non-functional requirements delivered to the implementation team with little to no detail. Here are a few of the worst but most common examples:

• Security – The software must be secure.
• Performance – The software must be fast.
• Usability – The software must be easy to use.

Most software professionals have been taught that non-functional requirements are important, but many projects skip over them in order to get to functional use cases and writing code. The result can be profound, leaving the implementation team without sufficient input to make critical design decisions that will be very costly to change when the non-functional requirement is later clarified.

What Every Non-Functional Requirement Needs

For every non-functional requirement, the software architect should assure that the following questions have been adequately answered.

• To whom is this quality important?
• Users and integrators
• Management team
• Implementation team
• Operations team
• Who will assure this quality is met?
• Implementation team
• Operations team
• Management team
• How will this quality be met?
• Cross cutting constraints in software
• System and network constraints
• Log analysis and oversight
• How will we know this quality is met?
• Scenarios with measures
• Monitoring and review
• Acceptable tolerance percentiles

The items below each question are not meant to be an exhaustive list but simply to give you an idea of what may be involved in answering those questions.

Classification of Non-Functional Software Quality Requirements

Clarifying and prioritizing non-functional software quality requirements may be easier when you classify them into one of four groups by answering two questions: operational or non-operation, and internal or external. The following table is anything but exhaustive but it will give you the general idea.

 
Business stakeholders are generally more interested in and will support efforts to meet external qualities. Implementation and IT teams sometimes have to work a little more to garner support for time and effort and expense for internal qualities.

It is often easier to build into an implementation the cross cutting concerns to measure operational qualities. Collecting performance, reliability and security metrics from executing code is always possible with well planned constraints early on in the development effort. If these qualities are defined later, the refactoring process can be challenging.

For non-operational qualities, other systems such as those used to manage support issues and ongoing development efforts are often helpful in measuring the cost of change to the system or whether usability goals are being met. Sometimes time series log analysis can be utilized to extract measures for non-operational qualities, especially those most important to external parties.

Use an Agile Approach to Non-Functional Requirements

However you choose to collect and document non-functional software quality requirements, you should continue to improve and tweak them throughout the development process just as you would with functional requirements, grooming your backlog and prioritizing based on ongoing feedback from stakeholders, users and developers.

I have been enjoying an e-book called Software Architecture for Developers by Simon Brown that was very well worth the price. I also just finished watching the author’s presentation at the 2014 GOTO Conference. A very thought provoking presentation.

Here are a few things that I like very much from the book and presentation.

• “The code is the single point of truth. It is the embodiment of the architecture.”
• TDD does not replace architecture. Do TDD inside a set of boundaries and frameworks provided by the architecture. (see Why Most Unit Tests Are a Waste by James O Coplien.)
• If the diagrams don't reflect the code, the diagrams are basically pointless. We're just deceiving ourselves.
• Component testing is preferable over unit testing but unit tests and mocks for testing against async systems are still useful.
• Layered architecture can lead to a big ball of mud because too much functionality is exposed for public use by other layers.
• Component organization over layered is preferred. Components have limited public interfaces (aka ports) and may have layers within the component that are not improperly publicly accessible.
• If a system has a very large number of unit tests, we may have an out of control layered architecture.
• “If your software system is hard to work with, change it. This is entirely within your hands.”

C4: Context, Containers, Components, Classes

I especially like Brown’s use of what he calls C4. In essence, every software system can be broken down into a simple hierarchy:

• Contexts (systems) made up of
• Containers (web server, app server, database, browser, file system, etc.) which host
• Components which expose one or more interfaces (sometimes referred to by others as ports) and contain
• Classes which implement those interfaces and the layers behind them.

By creating architecture diagrams that follow this hierarchy, it is possible to create code that matches it. It is an architecture that developers can use directly. (Side note: I have often used a Visual Studio solution to lay out a project in similar terms directly in code. I have also gone down the layered road only to regret it later and end up pulling those layers apart and encapsulating them into a component-like approach to ensure that the responsibilities and behaviors exposed to the outside world are used properly and that the system remains well ordered.)

Here is my simplification of what the author already makes rather simple with respect to these constructs. I recommend you buy the book and get full details and example diagrams, but the ideas are what is most important.

• Context Diagram - A context diagram answers what are we building, who is using it and how does it fit into our IT environment and business. It is important to note at a high level how users will interact with the system.
• Container Diagram - In the container diagram, you define the shape of the software, high-level technology decisions, how responsibilities are grouped and separated, how each container communicates with other containers, and where the code lives for each container.
• Component Diagram - The component diagrams answers what course grained building blocks of functionality are required, what are their responsibilities, and how will they interact with other components (interface and communication mechanisms such as sync, async, batched, etc.).
• Class Diagram – Brown does not explicitly discuss this level, as far as I’ve read. I assume this is because it is rather intuitive to most architects and in part because the design at the class level is often better left to the implementation team. However, from my own perspective, the architect should consider defining the public interfaces and primary classes to name and separate key responsibilities into distinct code structures that will be intuitive for the implementation team to complete—naming conventions that separate what may be thought of as traditional layers and that the implementation team will understand is crucial here.

Some of what Brown proposes breaks with what others may consider traditional software architecture. I am excited about Brown’s challenging of the status quo, questioning our assumptions. I believe his ideas will help us narrow the model-code gap (see Just Enough Software Architecture by George Fairbanks—one I’ve just added to my Kindle collection for some fun future reading).

Measurable Non-Functional Quality Scenarios

Brown covers the importance of quality scenarios, but on this topic I prefer the more pedantic but measurable approach of Len Bass in Software Architecture in Practice. I believe his emphasis on defining metrics driven quality scenarios is well worth pursuing, especially to the point of implementing logging and monitoring systems that allow you to constantly measure non-functional quality and improve against those measures. Here’s the structure that Bass recommends:

• Source – where does the input come from?
• Stimulus – what is the input?
• Artifact – what container and component are affected?
• Environment – in what context does this occur?
• Response – how and with what did the software respond?
• Measure – how much time, how many clicks, how many errors, were proper notifications were sent?

Creating quality scenarios that developers understand and can incorporate into the software they are building is critical to long term success of the software. To paraphrase the quality gurus, if you can measure it, you can understand and improve it.

Most important, if your implementation team and your stake holders understand your architecture diagrams and documents, you are more likely to succeed. And if your code does not mirror your diagrams, no amount of code reviewing will tell you whether your architecture has been implemented. And that is Brown’s greatest point in my opinion.

Platform or technology stack specific modifiers on the title of Software Architect are common. Most software architects know that the platform, like a framework, is an interchangeable implementation choice and not really part of the software architecture (see my post entitled Practical Agile Software Architecture).

Why the distinction then? Two reasons:

• Architect as Implementer  - Many software architects are involved in guiding and contributing to platform specific implementation which requires specialization that is not specific to the architecture itself. I prefer this combination.
• Platform Specific Language - The platform specific software architect may be most comfortable or even required to produce artifacts that adopt the vernacular of that platform in order to make those artifacts more easily consumed by the implementation team.

The most common platform and implementation specific language elements in architecture artifacts replace the more generic terms of module, component, and software. Here’s an overly simple conversion table that I have found helpful for Java and .NET.

.
Connectors are generally common across these stacks, with a few exceptions such as WCF and JAX. More often connectors are literally specified as a SOAP or REST or even a custom TCP based protocol. Message based connectors are very often technology specific, indicating the specific message queue but be careful to avoid limiting your architecture by specifying implementation choices. The implementation choices should be, as much as possible, left to the design and implementation team effort.

The majority of my implementation experience has been in the .NET stack. But software architecture should be the same across implementation stacks. Of course, there may be some things easier said than done in a specific platform and technology stack, and that may have an impact on your software architecture choices, but minimally so.

Ask software professionals what software architecture is and you will get many different answers.

In the past I have worked as a software architect and development lead on a variety of projects and teams. Two years ago I joined a larger software development organization as an individual contributor, and for the last eight months I’ve been the sole engineer assigned to create a very large in-memory graph data service that provides sub-second traversals across tens of thousands of edges amongst several billion. It has been a fun and successful project, but lately I’ve been thinking more and more about software architecture.

I’m sure that millions of words have been written on the subject of software architecture. I have read of them. Most recently I’ve been perusing two tomes on the subject: Software Architecture Foundations, Theory and Practice (Taylor, et al) and Software Architecture in Practice - 3rd Edition (Bass, et al). The latter is quickly becoming my favorite, but both of these are exhaustive academic textbooks, so most software professionals would find it impractical to put them to good use. Even so, I recommend them to you.

So what do I mean by “practical agile software architecture” and how can it be used by an agile software development organization? That is the question I will attempt to answer here. Distilling some of my reading through the filter of my own experience, I offer the following thoughts on the matter.

Principles of Practical Agile Software Architecture

• Architecture is part of the product or a product dependency – Architecture should come first. Any large and complicated software project should have a architectural artifacts (see below) sufficiently defined to guide product backlog story definition to seed and sustain ongoing implementation with minimal risk of disruptive architectural changes.
• Architecture stories go on the backlog – Architecture stories are generated from the stakeholder/view matrix (see below). Stories may repeat by adding detail to or modifying existing artifacts. Modifications should therefore be taken carefully considering the implementation changes that may be required.
• Architecture story implementation produces artifacts (scenarios and views) – The architectural scenarios and views (Bass) include the definition of non-functional quality attributes such as security, reliability and performance. These artifacts document the architecture sufficient for the implementation team as well as stakeholders.
• Architecture quality attributes should be measurable – Simply declaring that the application must be fast is not enough. Quality attribute definitions, as one type of artifact (see scenarios below), must produce measurable, observable, testable indicators.
• Architecture artifacts evolve iteratively – Each iteration should produce a useable architecture, sufficient to inform stakeholder decisions and guide implementation. This does not mean that the architecture is never revisited. Architecture should be a regular part of every iteration, even if the story is simply to review some part or the whole of the architecture that is relevant to other stories being worked on during that iteration.
• Architecture change can affect product backlog – Architecture stories that result in a modification to an artifact will generally produce one or more changes to the product backlog in order to accommodate the modification in product implementation. Care should be taken to communicate each new version of the architecture to relevant stakeholders and the implementation team.
• Architecture complexity should be decomposed – In multi-team multi-service multi-product development organizations, it may be useful to create a centralized architecture team that defines and develops higher level architectural views upon which implementation teams take a dependency. Changes to the central architecture may produce changes to a service or product’s architecture related stories on their own backlog. As scrum of scrums can help to decompose more complex projects, so can the same approach be used to break down the complexity of the overall architecture to ensure agility.

Architecture Artifacts and Views

Documenting an architecture can be challenging. Some documents are produced as an overview. Some with varying levels of detail. Much of this depends upon the extent to which the architecture is complete and the intended audience for the specific view. Whether you call these documents a “view” or a treatment or a physical or logical or quality requirements document, the result should be the same. These documents should convey as concisely as possible the information needed to by stakeholders and implementation teams to make decisions.

These are summarized from Bass with my own particular spin.

• Qualities Scenarios – There are seven primary non-functional software qualities: security, availability, interoperability, modifiability, performance, testability and usability. These should be well defined with concrete scenarios specifying stimulus source, stimulus, environment, artifact, response and acceptable response measures. Here’s a summary example for each of the seven qualities.
• Security – A user (source) attempts to view private information (stimulus) in the patient records systems (artifact) on the beta testing system (environment) and an audit trail is maintained (response) and the attempt is prevented and reported (measure).
• Availability – A heartbeat monitor (source) receives server unresponsive status (stimulus) on a web service (artifact) in production operations (environment) and informs the failover system to take traffic (response) and no downtime is recorded (measure).
• Interoperability – Our patient billing system (source) sends account to collections (stimulus) to the collections system (artifact) in the testing system (environment) and a test letter is sent (response) and the test letter queue increases by one (measure).
• Modifiability – Developer (source) is required to change a data source (stimulus) on the records repository (artifact) in the source code (environment) and makes the change with a unit test (response) and completes the task in three hours (measure). 
• Performance – A user (source) save changes to his profile (stimulus) in the company intranet (artifact) on the intranet (environment) and the change is displayed (response) and the change takes less than two seconds (measure).
• Testability – An automated unit test runner (source) executes a unit test (stimulus) on the code (artifact) in the test server (environment) and the test passes (response) with 85% code coverage (measure).
• Usability – A user (source) adds a widget to his blog (stimulus) on the admin page (artifact) in the production system (environment) and is taken to a page with the widget activated (response) with fewer than 3 modifications (measure).
• Module View – In the module view, design principles such as separation of concerns and loose coupling help to define boundaries around responsibility sets. Often a module is a separately compiled unit (e.g. a .NET assembly DLL).For example, the data repository module depends on and implements the data repository interfaces module which provides data models and the interface definitions for persisting those models. The data repository module can be replaced when a decision is taken to move from one database server to another.
• Elements – module = implementation units (sets of responsibilities)
• Relations – is a part of X, depends on Y, and is a Z 
• Constraints – visibility and availability of module
• Usage – implementation blueprint, impact analysis, requirements tracing, information design, users stories and use cases
• Component & Connector View – Components and connections define logical processing units (processes or sub-processes) or data stores and their ports (sometimes called API) for connections to connect to. This could be a web service and the port (not TCP port) that defines the external boundary of that processing unit. A connector may me a message queue pushing and pulling messages or a browser connecting to a RESTful web service.
• Elements – component = processing unit or data store, connector = paths of interaction (external API and use of same)
• Relations – attachment = connector connected to a component port, interface delegation = a connector between compatible ports
• Constraints – components attach via port only to components and vice-versa, never to each other
• Usage – illustrate how the system works, define communication boundaries, guide implementation and design structure, and guide establishment of quality attributes for each component
• Physical Allocation View – Define the software that will run on each part of the physical deployment in each environment.
• Elements – software = what runs on each part of the environment, environment = where software is hosted and on what
• Usage – assists reasoning about quality attributes, distribution of work and teams, concurrent access to versions of the software, and the form and mechanics of software deployment
• Qualities View – A quality view document helps to correlate specific qualities and their scenarios with respect to the following:
• Security – define components and behaviors that have an impact on security and how security risks will be mitigated and prioritized.
• Communications – document how components will communicate, what networks channels and internal and external systems will be involved in data transfer, and how quality of service (QoS) will be maintained and how communication interruptions will be managed
• Exception / error handling – define how errors will be detected, reported, monitored and resolved
• Reliability – define how timing, integrity and other quality measures will be recorded and monitored
• Performance – establish traffic load expectation and handling patterns, latency and system and network metrics

For more complete details on how these views may be composed, I recommend picking up a copy of Software Architecture in Practice as previously mentioned.

Stakeholder View Matrix

Because the number of possible views and scenarios with varying levels of required detail can overwhelm the architecture backlog, I endorse the Bass recommendation of the stakeholder/view matrix. I’ll try to summarize it here.

Create a matrix of stakeholders, including implementation and testing roles, with common architecture views and qualities. Let’s assume you have executive, product manager, implementation and testing stakeholders. And you have seven qualities and a minimum of four views. This gives you a four by eleven matrix.

For each cell in the matrix enter a level of detail word (none, overview, moderate, high) and a priority number between 1 and 10. Use this matrix to identify artifacts that need to be produced first and what level of detail is required for the first, second and third iterations. This exercise will help you identify the most important architectural stories for your backlog and prioritize your work on the architecture.

Thoughts on Architecture and the Role of the Architect

Much of what a software architect is often required to do is not, strictly speaking, architecturally related. But if an architect wants a team to be successful at implementing the software, she or he will take special care to incorporate these things into their regular work.

• Patterns and Frameworks – architecture is NOT a framework. In fact, a framework should be able to be swapped out without really having to change the architecture. Frameworks are tools and as such should be carefully chosen for their implementation multiplier capabilities as well as their simplicity and adaptability. The same is true for implementation patterns. Developers who are guided to select appropriate implementation patterns by being provided prototypes and samples can be far more productive and will produce much more maintainable code. The effort invested by the hands-on architect in these will yield handsome returns.
• BDD and TDD – architects often wear the design hat as well and when doing so, they should adopt Behavior Driven and Test Driven Development. Read the linked content. It will really help simplify and clarify your user stories, use cases, and test designs. Of course, most design is driven by functional requirements which can affect the architecture in order to support the behaviors defined by the functional requirements. The software architect is appropriately involved in gathering and understanding these requirements. And breaking them down by behavior driven forms is an ideal way to fully understand and document these requirements.
• UML and Other Modeling Languages – there are many tools available to create diagrams and other artifacts that may help you communicate your architectures and designs. A word of caution. Unless your audience is familiar with the notation and the meaning of the symbols and shapes you use, you need to provide a key with sufficient detail that a less familiar reader can understand what it is your are trying to communicate. If the diagram uses too many abstractions, the reader, even a seasoned software engineer, may have trouble understanding your meaning. And when stakeholders and implementation team members do not understand it, they will not always ask. Most of the time they will either ignore or or make erroneous assumptions that will bite you in the tender flesh later.
• DevOps and Infrastructure – one of the things that can kill a great architecture and implementation is the lack of sufficient means to build, test and deploy, and roll back when necessary, in an automated fashion. A great DevOps team can make or break your large software development organization. And if you have the necessary operations monitoring and SLA failover triggers enabled with redundancy, you can maintain a very high level of service and reliability. A software architect’s input into the creation and improvement of these is critical to success.

Conclusion

Software architecture is not necessarily easy in part because we often make it more complicated than it need be. It is also true that we rarely define it well within an organization. Many organizations hire and promote to the position of software architect with the expectation that they will be an experienced software engineer who has picked up on what works and what does not and will help lead a team to success. Many experienced software engineer’s make this transition and learn the craft of architecture well. Some do not.

Some software architects spend very little time defining and refining the actual architecture of a the software. Instead they spend far more time chasing down design flaws, resolving thorny implementation issues and untangling behaviors that are at odds with business requirements. And yet, had the architect spent sufficient time and energy producing concise and precise architecture artifacts discussed in this post, continually improving them, reviewing them and coordinating their impact on implementation backlogs, many of the problems they spend so much time on would have been avoided.

I know you have differing opinions. Please share them here.

While browsing the deep space of Alpha Quadrant of the web this evening, I ran into a Forbes article entitled, 6 Fundamentals That Can Make You A Better Manager In 2014 by Victor Lipman. I enjoyed the brief article so much, I decided to refactor it into my own words for the manager we either want to be or want to have.

In our agile quest to improve our software and our processes, we may sometimes overlook the nuances of management. In shops with empowered agile teams, it is possible for managers to make management an afterthought, allowing the self-organizing teams to pick up the slack. It is my opinion that this false sense of security and resulting dip in the quality of management can lead to fundamental and long term organizational and cultural debt.

To avoid accumulating this management debt, team members should encourage, perhaps even require, the following from their managers. And certainly managers should strive to focus on these fundamentals even while riding on the success of an empowered, agile development team.

1. Be open to suggestion. Seek out and embrace ideas and opportunities to improve your management practices. Be a good listener. Take regular one-on-one’s with your direct reports. Conduct “town hall” meetings with your direct reports and their teams. Encourage honest and open feedback. Don’t allow the “way it’s always been done” to overshadow your improvement courage.
2. Expect excellence and reward it. Set high but attainable expectations. Communicate them clearly. Be gentle but firm and require regular accountability and reporting. Openly recognize and praise success. Privately discuss under-performance and obtain solid commitments to improve from under performing members of your team.
3. Use your time effectively and efficiently. Be generous but measured with your time. Spend small amounts of planned time socializing with your team members. Be careful not to impinge on their time or waste yours. A thirty second conversation can do wonders for interpersonal rapport but a five minute chat session can degrade efficiency. Protect your schedule. Insist on well run, timely meetings. Focus on your priorities while making a top priority of maintaining a personal connection with your team. If they know you care, they will care when you need them.
4. Communicate feedback in realtime. Your team needs regular and immediate feedback. This includes public praise for a job well done. It also includes private and direct feedback on unfulfilled expectations with an immediate call to corrective action or a resolution to an expectation that has become unreasonable. DO NOT do negative feedback by email. Back up any positive emailed feedback in person. Real words from a real person mean a thousand times more than an email.
5. Embrace conflict. Don’t run away from or duck conflict. Hit it head on, in person, and resolve it. Don’t dwell on blame but fairly examine cause and effect and then focus on actions required to resolve the conflict and move forward. Don’t be afraid to apologize or accept responsibility for creating conflicting expectations or misunderstandings. Invite others to suggest ways to improve. Listen and see step #1.

Now go be a better manager. And if you want a better manager, share this or the Forbes article with her or him.

I am very excited about the .NET Native project. And while this is just for WinRT app store apps right now, I was encouraged by the commentary in the Channel 9 presentation. I am eager to learn more about it.

For a long time, the speed of desktop processors has spoiled us, allowing us to depend on heavier and heavier JIT workloads and dynamic utilization of multiple core libraries. With the challenges of mobile coming along, we get the benefit of heavy duty platform specific optimization in .NET Native now.

Because I often write server applications in C# that could really benefit from that extra platform optimization, I am hoping and looking forward to being able to utilize .NET Native in my work as well.