Does the architecture matter when the ways-of-working change?

More and more embedded companies adopts agile development as their way-of-working for software. For small systems this is straightforward, but for large development projects with hundreds of developers working on the same product there seems to be a typical evolution that an organisation goes through before successfully scaling agile.

Typically the transition starts with individual teams adopting agile practices, but fast iterations and shorter lead-times on team level does not automatically lead to shorter lead-times for new or updated products. The organisation as a whole still uses a stage-gate process or V-model for the product development. If the organisation is successful in changing their entire ways-of-working across all departments it becomes evident if the product architecture supports or hinders the new way-of-working.

The presentation concludes with some general guidelines for how software should be designed to support agile development of large embedded software systems.

Is “process” the best metaphor for software development

I think the phrase “development process”, or rather the concept of development as a process has become a paradigm in software. A google scholar search on software development process results in 2 650 000 hits!

The view of creating software done through a “development process” is so established, at least in the software industry, that no-one seem to question if this is a valid view on how to design stuff. This regardless if one talks about waterfall, spiral, agile or lean.

For me a process is a repeatable set of activities or process steps that leads to the same results. typical examples is the process of producing paper form wood or assembling parts to a car. And much effort is spent on making this process as efficient as popsicle or achieving the least possible variation in output quality. Since I am not a native English speaker it could be a lack understanding the connotation of the words, but I’ll try to explain what I mean.

First, software is developed, but no-one talks about a software creation process. To me development is more akin to refinement, exploitation or progress. I think software is created, it is one of the very few intellectual pursuits, besides the arts, where something is created from nothing.

Second, the development (or creation) is done through a process, something which in an engineering perspective means that by combining the right inputs and having the right circumstances, a desired output is achieved. And the process is repeatable as long as the inputs and circumstances are the same. I don’t think anyone disagrees when I say that not two software projects are alike, regardless how similar one tries to keep the process.

So the question is how should we view software creation? I think it would be helpful to see it as a creative endeavour similar to what a writer does. There is usually some idea of what the end result should be, but that may or may not be very detailed when starting to make sentences. Some write in a very linear fashion, starting with the first chapter. Some start with an outline which gets more and more refined until it actually is the full text. Some writers are very disciplined and can write for eight hours day, some can just write when they are inspired. And everybody has heard about “writer’s block”…

If software is not developed in a process, but seen as a work of creativity I think efforts should be to make the people writing software as creative as possible (and therefore also productive). Maybe this can be used as a “lean” principle on how to manage a software project.

Is there any use of the V-model?

This blog post originated with some thoughts when I attended the 13th International Conference on Agile Software Development in Malmö Sweden. As usual when I listen to presentations these tend to trigger new liner of thoughts instead of pondering the details of what was said. I guess this is a flaw in how I internalise knowledge from others.

Not surprising, nobody here mentioned the V-model since this is a conference e for those already embracing the gospel of agile development. Too bad, since I think the V-model is one of the most important metaphors regarding system (and software) development there is. But it is also on of the metaphors that have been used in ways which are completely inappropriate.

My “condensed understanding” of the strengths of the V-model is this: It defines a set of activities on the left side of the V with associated artefacts aimed at exploring the problem and detailing the solution down to the actual product (the code!). The important part is that these design activities have corresponding activities of verification & validation on the right hand side of the V, in a one-to-one relationship between the design and V&V activity. The number of levels is individual, but usually about 3-5, including code is reasonable.

Where it goes wrong is when the V is laid out in time as a template for progress of a project. This may make sense if you are designing for manufacturing-heavy products, but is useless for software, the time between understanding the problem (top-left part of the V) and where you verify that your implemented understanding is correct is just way too long to be competitive. There are also drawbacks since when project schedules are tight it tends to squeeze the V&V efforts on the right hand side of the V. This is definitely not news for software developers, and is a key point in agile software development (but probably phrased differently).

It seems to me that in order to optimise the efficiency and speed there is instead a tendency to downplay the activities above the point of the V ( the working code). Big-upfront-design is seen as such a bad thing that a projects ends up in with no design at all, there is a direct jump between formulating the problem (e.g. user stories) and start coding. I believe that in an agile project all activities in a V must be executed in each sprint in order to claim to be “truly agile”. Each sprint means a better understanding the problem, a better design to solve it, and a better implementation in the code. Of course all of these are also verified and validated to the required quality or acceptable level of risk. That does not say that these activates are equally large, or the size of the V is the same in alls sprints, but if you leave out the conscious architecture in the sprints you don’t learn to do better in the next sprints.

You should design the sprint activities in the V to provide the maximum added value with respect to the spent effort (which obviously depends on thousands of context-specific factors), but saying they are irrelevant is only reasonable for the simplest of systems, where you have a definition of the problem and code.

Personally I rather define the result of the activities, i.e.the artefacts, and let the teams decide themselves on how to populate them. I know blog readers may cry about excessive documentation, but what I am talking about is the necessity to externalise information necessary to share between developers, and teams, and to preserve this information over time, something which code never can do.