Mike MacDonagh's Blog

Somewhere in the overlap between software development, process improvement and psychology

Tag Archives: uml

What is enough agile architecure?

I wasn’t planning on writing a “how long is a piece of string?” post but it’s a question I often get, and something that I’ve played with a bit. The point of architecture is to address the aesthetics of a system, to consider its reusable bits or common forms, the overall shape and nature, the technology it’ll use, the distribution pattern and how it will meet its functional and non-functional requirements.

Of course in an agile, or indeed post-agile world, we don’t want to spend forever document and designing stuff in analysis paralysis. I’ve worked in projects where I had to draw every class in detail in a formal UML tool before I could go and code it. I’m pretty sure this halved my development speed without adding any real value. But I’ve also worked on projects where we’ve drawn some UML on a whiteboard while discussing what we were going to do and how we were going to do it – and that was really valuable.

This makes an architect’s job difficult. Of course, it’s always been hard:

The ideal architect should be a man of letters, a mathematician, familiar with historical studies, a diligent of philosophy, acquainted with music, not ignorant of medicine, learned in the responses of jurisconsults,  familiar with astronomy and astronomical calculations.

Vitruvius ~ 25 BCE

But as well as being a bit of a Renaissance man an architect also needs to know when enough is enough. I’ve found that I’ve done enough architecture with the team (not to the team) when we collectively feel like we understand the proposed solution, how it’s going to hang together, how it will address the risky bits and meet it’s requirements.
To do that, we tend to draw a few diagrams and write some words.

First, an architectural profile that gives us an idea of where the complexity is and therefore where the technical and quality risks are.

Second an overview sketch that shows the overall structure, maybe technology, target deployment platforms and major bits.

Third a set of lightweight mechanisms that cover the common ways of doing things or address particularly knotty problems and address some of those risks. These tend to describe the architecture (or mechanism flows) by example rather than aiming for total coverage.

I might add some other stuff to this if the project calls for it, like maybe a data model, a GUI mockup but generally that’s it 🙂

This post is an extract from the Agile Architecture content from Holistic Software Engineering

Scaled Agility: Architectural profiling

Architectural Profiling is borrowed from Holistic Software Engineering

When it might be appropriate

  • In situations where a lightweight approach to intentional architecture is required
  • In situations where high design formality isn’t required
  • When a simple approach to architecture analysis is required at a team of teams level before more analysis in contributing teams
  • Where a team wants to cut wasteful requirements and architectural “analysis paralysis” without throwing out ignoring technical risks
  • System of systems development

What is it?

When I look at a potential (or existing) system I think of it in terms of it’s complexity in terms of a few dimensions, they’re not set in stone and I might add or remove dimensions as the mood, and context, takes me.  Doing this early on gives me a feel for the shape of a project’s requirements, architecture and solution. In fact it also means I can short cut writing a whole bunch of requirements, acceptance tests, designs and even code and tests.

Here’s an example of thinking about a simple-ish app that does some fairly hefty data processing, needs to do it reasonably quickly but not excessively and has got to do some pretty visualisation stuff. Other than that it’s reasonably straight forward.

You might notice that the x-axis is pretty much FURPS with a couple of extras bolted on (I’ll come back to the carefully avoided first dimension in a minute).

The y-axis ranges from no complexity to lots of complexity but is deliberately not labelled (normally one of my pet hates) so we can focus on the relative complexity of these dimensions of the requirements, quality,  architecture and therefore solution.

The height of one of these bars helps me shape the architectural approach I’ll take to the project, and which bits and bobs I can reuse from my super bag of reuse.

Read more of this post

Lightweight architectural mechanisms – specification by example

In my previous post I talked about using a sketch to describe architectural structure, but the other part of a useful architectural description is it’s dynamics, best expressed as architectural mechanisms.

Mechanisms are little snippets of the architecture that address an important problem, provide a common way of doing something or are good examples of how the architecture hangs together.

Mechanisms exist within the context of an architecture, which provides overall structure for the mechanisms. I tend to use a simple architectural overview sketch to do that and then further refine the architecture, if necessary, in terms of mechanisms according to the architectural profile and (during development) the needs of my team.

Sometimes during a project the team will comment on the need to have a common way of doing something, or that they’ve uncovered something tricky that we need to consider as a more significant part of the system than our early analysis showed. In these cases it’s time to create a mechanism.

Mechanisms are great, but you don’t want to many of them, or to document and detail them too much, just enough to communicate the architecture and support maintenance efforts. Indeed writing too much actually makes it harder to communicate.

Mechanisms are best expressed in terms of their structure and behaviour, I tend to use a simple class diagram for the first and whatever seems appropriate for the second. This might be a UML sequence diagram, but I don’t really like those, instead I might use a good old fashioned activity diagram, or a flowchart with GUI mockups in the nodes. Either way I recommend limited the documentation and description, just because one flow is worth writing down to explain it the others might not be. In this way I do architectural specification by example. Once I’ve written enough about a mechanism that the rest can be inferred I stop.

The words aren’t important in this example but you can see that I try to fit the description into a fairly small concise area – that helps me focus on just the really important stuff. In the top left there’s a list titled “Appropriate for stories like:” which is an indicative list of a few things to which the mechanism is appropriate.  Next to it is some blurb that says what it’s for and the main scenarios it covers, so in the case of persistency it’s the normal query, create, edit, save & delete. There might be some notes around important constraints or whatever else is important.

I’ll then describe each important scenario in terms of it’s behaviour in whatever language or visual form makes sense. Sometimes  this is a photo of a whiteboard 🙂 Sometimes it’s text, sometimes it’s a combination of those things.

The flip side

Just like stories having a flip side which contains their acceptance tests I also like to put acceptance tests on the flip side of my mechanisms. Although many are easy to frame in terms of customer acceptance tests (e.g. Search Mechanism will have performance, consistency and accuracy acceptance criteria) some are a little harder to frame. Technical mechanism formed to provide a common way of doing something in an architecture or to express the shape and aesthetics of an architecture may feel like they only make sense in terms of the development team’s acceptance criteria, however I always make sure they relate back to a story if this is the case, otherwise I could be needlessly gold plating.

Mechanisms are best found by understanding the architectural profile initially and then by actually building the system. If the customer doesn’t have a story that will be satisfied in part by a mechanism then it probably shouldn’t be there. Even if it is shiny.

Lightweight architecture sketch in a single diagram

This blog is based on Architecture in Holistic Software Engineering.

I’ve been doing architecture for a while, in fact it’s what I used to do as my main job. I’ve taught UML, Object Orientated design and coding and various bits of various processes for years. One thing that’s stuck me over the years is that most of the descriptions of how to capture and communicate architecture aren’t simple enough.

I quite like UML, it’s useful to be able to draw a symbol and others know what it is without me having to explain to everyone what I mean, but I don’t like the way it has so many restrictive rules that stop me from making a nice sketch to explain what I mean, also everyone else doesn’t seem to know the language to the same degree, I need something a bit lighter.

I don’t want:

  • to be limited to the symbology of UML
  • a lot of model structure with interconnected diagrams
  • endless detail
  • every class on the diagram
  • to follow all of the rules
 I do want:

  • the symbology of UML
  • the important elements
  • to give a feel of the important structural, logical and physical stuff
  • just one diagram

I’ve always done a high level diagram that shows the overall pattern for my architecture, something like layers or pipes and filters or whatever. I’ve also always done a breakdown of the important stuff within each layer but I’ve had the best success (in terms of communicating with others) when I’ve mixed both, with elements of the target deployment and actor interaction.

Being terrible at naming things I call this marvellous diagram the “Architectural Overview Sketch”. Here’s an example:

It  expresses all of the structural things that I care about. It expresses the:

  • the shape and feel of the system
  • high level layers
  • primary interfaces between subsystems
  • target client platforms
  • User – GUI interaction paradigm
  • important classes in each layer and major layer package structure
  • critical data schema
  • interaction with external APIs
  • the middleware and database hosting and distribution

I might have more diagrams to explain more structure in part of this if it’s really important, but I don’t want to have every class in my system on a diagram somewhere. I’m using my diagrams to communicate, not specify. I’m broken a bunch of UML rules of course, and there’s a lot of implied stuff but adding those details makes it harder to explain what I really want. One thing I really like about it is how it shows the important detailed parts of a design in the context of the bigger architecture.

Architecture, like any design, is best expressed in terms of both structure and behaviour. So far this is just structure, there’s some hints at behaviours but nothing terribly useful. My next post will be about how I minimally specify the important bits of an architectures behaviour – the mechanisms.

Zen and the art of Enlightened Software Development

When I started doing software I was a simple developer interested in elegant code and shiny things. I worked in a small software house that taught me many bad practices in terms of configuration management, change control, estimation, management etc. It was an incredibly valuable experience for me in learning how not to do things and led me to strive for something better even though at the time I didn’t really know what it might be.

That led me through a path of process definition and documentation (RUP) and a rather limited following of an iterative lifecycle. I remember a conversation I once had with one of my project managers (when I was an architect) asking if we were going to do “Use Case Analysis” on a particular project or just skip straight into “Use Case Design”. The question puzzled me then because it felt like a trap and I didn’t really understand what I was being asked.

It puzzles me even more now as I know understand “analysis” to be a stop-and-think-for-a-moment activity and I’d always do a bit of analysis of my requirements, however these days it’s extremely unlikely that it’d be an analysis UML model, instead it might be a sketch, a conversation and a bit of thinking before another conversation.

So I moved from process prescription to process understanding, applying the spirit of doing things well rather than the letter of whatever current process law was in fashion. Following on from that I’m not really interested now in the details of what a book says someone’s role should be or how people should interact instead I think the real challenges in software development are social, not collections of technical practices (although there is  value in evolving better practices and tooling).

A colleague of mine commented recently that 20 years ago when he was doing software it was possible to understand everything from the metal all the way to the blinking lights on his bit of hardware. Software development in just 20 years has progressed incredibly and it’s just not that simple any more. We can understand the basics all the way through the stack but not all of the details. There are so many bits interacting that it’s just too complex. As a result the problems, technology and team working are all abstracting away from the tangible mechanistic past.

Developers are on a path from technical skill to mastery, as they begin to understand the kung fu of software engineering they can apply experience and deep understanding to solving the technical problems, doing away with formal process and just using the practices that they intuitively need, happily breaking the “rules” to get the job done in an efficient high quality way. I’m not entirely sure where brogrammers are on this evolutionary path, I’ll leave that up to you.

The problem, if it even is a problem, is that each person in an organisation is somewhere different along this path and even if they’re at the same point in the same dimension they might not be aligned in their interests and motivations. This makes team working amongst inherently complex social creatures a tricky proposition. The sweet spot is a team of fully enlightened software kung fu masters but that’s a really hard target to meet for a number of reasons. Consider the flower of team working evolution weirdness, which area is your team in?

One reason that this sweet spot is difficult to hit is because in any organisation half of the developers are below average in technical and social skills. Sounds horrible but is obviously true and the larger the organisation the more likely it is to be the industry average, not the organisational average.

This means that the centre of skill gravity for any team is unlikely to be on the w00t side of the scale.

In most large organisations the teams are more likely to be flattened pancakes across this bell curve taking in a reasonable representation of the organisation as a whole (especially as the highest skilled are often distributed amongst an organisation to attempt to bring up other teams.

This isn’t necessarily a problem though as there is value in diversity and individuals will each have different ways of thinking about things that can bring value to teams and organisations.

I’m beginning to think that the purpose of classic software process, and classic software process improvement is to try and move people along this scale from basic developer to software ninja, helping them to gain mastery through experience and feeding in the experience of others. When a team moves along this scale they begin to not need their process mentors any more and will seize their autonomy.

Technology is increasingly commoditised by innovation, as is software development and software process. I used to spend a lot of time teaching people Object Orientation but these days everyone just seems to know it, it’s nothing special or new, it’s just what people do. Similarly people are increasingly aware of the value of iteration, limiting work in progress, continuous integration and delivery. They need less process and less instruction as this stuff is becoming business as usual at least in more mature organisations.

As these problems are being solved I think that bigger problems are coming to the fore as they’re less hidden by low level development issues. The questions I see a lot of clients wrestling with are things like:

  • How can we foster the right kind of organisational culture?
  • How do we deal with requirements, architecture, releases, resources etc. in a complex system of systems environment?
  • How do we bring together multiple divergent interpretations of “agile”?
  • How do we manage outsourcing contracts in high speed agile projects?
  • How do we motivate and engage the business and technical communities?
  • etc. etc.

For me doing these things in a social collaborative way, that values individuals and teams, is Enlightened Software Engineering and encompasses the buzzphrase of the moment: Agile at Scale.

Solving these problems is less about technical skills and technical process content and more about social skills, psychology and understanding. Both from a coaching and business perspective.

So what’s my point?

My point is that software process improvement needs to focus less on individuals and more on teams, and teams of teams. That we should avoid ideology and take the best bits of knowledge and experience wherever we find them, growing our teams and individuals.

Also that we should apply more psychology to software process and business change finding socially resonant patterns for how we do things. True mastery involves not worrying about “breaking the rules” and from the outside can easily be confused with ineptitude.

Microsoft VSTS, Camano and UML

I recently had the pleasure of going to visit the Microsoft Technology Centre to learn about all things MS for software engineering. Having long been a Visual Studio user and a VSTS (Visual Studio Team System) user since it turned up I was really pleased to see VSTS 2008 and some interesting stuff coming in 2010.

VSTS seems to be really maturing and now has a nive new non-sharepoint web UI. It will be interesting to see what happens in terms of Microsoft VSTS vs. IBM Jazz. I’ve written about Jazz before on this blog and haven’t gotten around to a detailed comparison yet but I was excited to see how mature VSTS is now looking and how nice it is to use. It’s designed to make developing easier and use the tools people already have in many cases. For example, using MS Project, Excel, Word, SharePoint etc.

Things of special note were the automated testing support (including coming soon recording and playback of GUIs other than web), the upcoming cool manual test runner, Camano for test management and test execution by proper testers not just developers doing unit tests etc. etc. Also checking out the project + project server for portfolio and project management views was interesting.

What I really really liked are the VSTS 2010 architecture tools. Microsoft has seen the light and embraced UML + DSLs (or DSLs + UML if you prefer) as the pragmatic modelling solution. I really like the idea of being able to do just enough  modelling, using the most appropriate language such as a high level architecture diagram that then imposes constraints on the code through things like check in policies!

I think this stuff is going to be really powerful and I can’t wait to get my hands on it properly (and somehow conjure the time from somewhere) to do some detailed analysis and also a comparison against competing products/toolsets.

Of course if you are using Visual Studio (or Eclipse) you can still use EssWork to use practices and processes directly inside your IDE

EssWork in Visual Studio

EssWork in Visual Studio

Interesting links in the world of software development

At IJI we have an internal newsletter than includes a section about software development industry news. For the most recent edition I collated a few links and I think they might be interesting for broader consumption. Of course one of the best links for Microsoft, Rational, Jazz and Borland information is my blog 😛

So subscribe

Here’s the links:

And an addition:

What’s new in terms of Rational Jazz products

%d bloggers like this: