Launch: Holistic Software Engineering
How do we join up business strategy to agile development? Is program management relevant? Where do project managers fit in? What about architecture?
Holistic Software Engineering (HSE) answers all of these questions – for free.
Agile and continuous flow are great for small teams or a small number of inter-related small teams working on exploratory or maintenance work. But what if we’re spending 100s of millions on an IT strategy of inter-related products that need to work together to deliver business value. What is business value anyway?
To answer these questions (and more) my friend Steve Handy and I have distilled our collective 30+ years of software experience in a single, cohesive model of software development. We’ve developed the H model that moves on from the v-model and it’s siblings by combining:
- Business strategy
- People and team issues
- Iterative and feedback loops
- Lightweight requirements and architecture
- Lean portfolio and program management
- Agile and continuous product delivery
- A focus on integration, quality and business value
…all elegantly combined and de-conflicted by release planning.
We’ve not invented very much, we’ve simply put a lot of good ideas from others together into a cohesive framework. We’ve drawn it all as a big picture and developed a website that shows how to get value from all of this stuff. Everything is clickable, everything has content.
The best bit: it’s free! There’s no paywall, there’s no private “full” version, you can just use it or not as you like.
We don’t believe in process zealotry, or putting academic concerns above clarity and usefulness. HSE is indicative, not prescriptive. You’re already doing it and if you use the big picture to draw red blobs on the bits that aren’t working well, or missing, in your organisation then you can use the model to make tangible improvements – immediately.
Using HSE doesn’t replace any of your existing processes, it provides the glue that joins them all up together in one simple, elegant and cohesive model.
Holistic Software Engineering
And if it’s too big, we’ve got a small picture, which is essentially “normal” agile development.
Please share HSE with your friends and colleagues.
No more Project Managers, bring in the Movie Producers
I was reading some course material recently that was trying to teach people something to do with software development and it was using the same old tired “ATM machine” example. I’ve worked with hundreds of projects, many in the finance sector and none of them are anything like an ATM machine. One of the reasons that example is soooo tired is that it’s describing commoditised software development, it’s something I’d expect to buy off a shelf or go to a specialised vendor for. It’s not something I’d put my team of uber |33t haxorz on.
Developing software products is a balance between a creative and scientific pursuit, so it’s a little hard to describe, especially when that software can range from a tiny smartphone app, to a website, to router firmware, to an enterprise hr system (urgh!), to a big data processing system of systems etc. You get the gist.
The things these types of system have in common with each other is how different they are. And yet the traditional approach to managing these diverse kinds of work has been classical project management with a one size fits all process (I’m looking at Prince2, RUP, Iterative, Agile etc.) or worse hiring a large company full of body-shop project managers. For me this is one of the root causes behind large scale IT disasters.
I once had a discussion with the leader of a PMO in a large organisation of thousands of people about the nature of Project Management for software and he assured me that “someone who knows how to manage a bit of work can manage software, it’s just a different type of work” and indeed I should “stop thinking of software as special”. I’ve seen this attitude resonate in many organisations and is to me best summed up by the same Head of PMOs statement “from a project management point of view building software is the same as building a bridge”.
Now then. I know a little about software development, but not that much about bridge building (if you exclude my illustrious Lego engineer career when I was 7). If I managed the building of a bridge next door to one build by someone with a track record of bridge building who’s bridge would you drive your family on? Not mine if you’ve got any sense.
There are so many problems in the software development industry caused by people not understanding it and applying bad metaphors. When people who do understand it start to get the point across (e.g. the early agile movement) they often get misunderstood by the old guard who dress up their normal ways of working in new language.
Many of our “best” metaphors come from manufacturing lines where the same thing is made over and over again. That’s nothing like software.
To me a software project is more similar to making a movie than anything else:
- It’s a unique one off product (or a remake of a previous version) with new parts working together in new ways
- Once made we’ll often duplicate and digitally distribute
- It can cost a lot of money because it can be very complex and needs lots of specialist skills
- You need a high level plan for how you’re going to make it
- There’s no guarantee on return
- What’s written down originally bares little resemblance to the finished product
- We know when it’s good or bad but it’s hard to objectively quantify
- There’s lots of different types that need different collections of specialist skills
- Both involve people and time and so are subject to change, they have to be adaptive
- You can tell when the people making it had fun
- It’s not feasible that any team member can fit in any role
- There’s almost always going to be some quality problems
- You wouldn’t get a movie maker to build your bridge
- You wouldn’t get a bridge builder to make your movie
- You don’t make a movie faster by telling the actors to act faster
So, I don’t want a Project Managers that know how to work a gannt chart. I want movie producers that know how to work with a team holistically to get the best out of them both technically and artistically.
Scaled Agility: The Project Forum
This blog is an extract from the Project Forum practice: Holistic Software Engineering – The Project Forum
When it might be appropriate
- In situations where multiple competing stakeholder groups with different agendas are required to work together
- In situations where multiple product groups need to collaborate on a bigger outcome
- Where there is a conflict in direction, resource management/ownership or scope between collaborating groups
- System of systems development
What is it?
The Project Forum is an application of agile philosophy to large project structures. Rather than impose a hierarchy of decision making from the Project Manager downwards the Project Forum is a virtual team in the middle of all stakeholders.
The Project Forum is a self-organising democratic group that balances competing voices and concerns, owns high level scope and architecture, runs the high level release train and performs integration activities for the product.
Use of the Project Forum practice does not prevent any communication directly between contributing groups it only provides a vehicle for that conversation when it’s relevant for the wider project.
The Project Forum practice is an example of Agile at Scale combining social business practices, technical software practices and ways of working to make a simple way of doing big complicated bits of work.
Decision making case studies
I’ve previously blogged on a decision making model that involves understanding the decision making process and then the different ways a group can reach agreement and how to choose which method to use in which situation. This post covers some real case studies from my own experience that apply this model and help discuss some the issues involved.
This post won’t make sense without reading How to reach agreement in a group – autocracy vs. democracy as it contains the decision tree and questions this post refers to.
How to reach agreement in a group – autocracy vs. democracy
In the first part of this blog (A model for making group decisions) I talked about the rational decision making model and how it is the basis of making a decision. Part of that model is gaining agreement amongst a group on an option out of a set of options which becomes the decision. This part is about different ways of gaining agreement and when to use each.
This model is a variant of the Vroom-Jago (1) contingency model in situational leadership theory first developed in 1973 and then refined in 1988. My take on it changes the language here and there to alter some of the emphasis and changes the decision tree a little.
This model defines a set of decision making models ranging from autocratic to democratic and then helps you choose which is appropriate for different situations.
Decision characteristics
This model applies to decisions involving a group that have an “owner”. The owner is the person who needs the objective decided on. The owner could be the customer, a representative, the group as a whole or the team leader. An important consideration when using this model, as discussed in part 1 is knowing who the owner is and realising that it may well be different for the different decisions the group has to make.
For each type of decision that needs to be made by a group:
- Identify the owner
- Clarify the objective
- Identify the team/group involved
Decision making models
The following decision making models are appropriate for group decisions:
- Autocratic 1 (A1) – The decision owner makes the decision based on the information available.
- Autocratic 2 (A2) – The decision owner requests information from the team (not explaining the situation or why they want information) and then makes the decision.
- Consultative 1 (C1) – The decision owner explains the situation to individual members (socialising and pre-integrating the decision) but does not convene them as a group, then makes the decision.
- Consultative 2 (C2) – The group discusses the situation and then offers ideas and suggestions. The decision owner then takes the decision.
- Group 2 (G2) – The whole group makes the decision with the owner acting more as a facilitator. Reaching this discussion can be discussion leading to emergent consensus, planning poker style or explicit voting solutions.
I don’t know why there isn’t a “Group 1″… maybe we should ask Vroom.
Which one to use
These options range from autocratic to democratic. One of the interesting things is that sometimes leaders want to be democratic but don’t really need to be or are not expected to be. This can lead to a conflict in expectation of approach. A leader may wish to be inclusive and democratic but their team may just wish they’d make a decision once in a while!
The alternative also causes conflict where a group expects their opinions to be heard and taken into account but the owner ignores their voices and makes autocratic decisions. This will create distance between the owner/leader and the group/team making the team feel undervalued.
Having understood the decision characteristics and the different decision making models you can then select one by using the following decision tree to identify the most appropriate model.
Each horizontal numerical bar indicates a question for the decision owner. Answer that question as honestly as possible, in terms of how relevant it is to making a good decision then move to the next part of the tree (sometimes skipping a question bar) and see which question you should answer next. Pretty quickly you’ll arrive at a circular blob with the short name of one of the options described above. I’ve written up some case studies to help explain by example.
- Are the stakeholders known, available and engaged?
- Are the people who will be materially affected by the decision identified, are they available to join in the decision making and engaged in the team to assist in making a decision?
- Is a high quality decision/solution important?
- Is this a case where lots of alternate options can be used and it doesn’t really matter which is selected? If so then answer “no”.
- As the owner do you have enough information available to make a gooddecision?
- If the owner is unsure, or wants to involve other opinions then answer “no”
- Is the problem well understood and does it have well known standard solutions that apply in this context?
- Is it’s a standard problem with a standard (or set of standard) solutions that will work in the current context then answer “yes”
- Do the members of the team or group have to accept this decision for it to work?
- If you (the owner) make the decision yourself will the group accept it?
- Answer this honestly, being in an organisational structure that means the group should accept it isn’t good enough. This question is about the real, honest dynamic between the owner and the group. This is affected by rapport.
- Are the group members aligned with the same motives and goals as you the decision owner?
- If the other members of the group have a different mission, agenda or motives then answer “no”
- Is disagreement likely among group members in reaching a decision?
What if you don’t want to do this model? What if your stakeholders object?
So if you’ve followed the decision tree and it says you should use Autocratic 2 (full autocracy) when you were hoping for Group 2 (full democracy) what should you do? This model is a bit like flipping a coin to decide something. The important thing is not which side of the coin comes up but how that decision makes you feel. If this process highlights to you what you were really hoping for then you’ve learned something. However it’s worth looking at the decision tree and seeing why you ended up where you did, and if other’s in the group would expect the same thing.
To explore this a little I’ve described three real examples here: Decision making case studies – please feel free to add your own. I’ve quickly written:
- Agile team customer sprint demo and assessment
- Process Improvement Team way of working decision
- Process Improvement Team scope agreement
The important part of this process is to make you, your team and your stakeholders consider the types of decisions they need to make, and how they should make different types of decisions. I recommend that as part of a team charter a team describes the types of decision it will make and how it will make them. That gives stakeholders, customers and other teams the opportunity to get involved and question the decision making dynamics if necessary.
Decision making as a team building exercise
During the formation (or reinvention) of a team you can do the following exercise:
- Brainstorm the decisions the group makes. Categorise them into a few groups such as:
- Achieving team buy-in to approach and activities
- Timebox assessment, regular reflect and adapt
- Balancing scope, cost, resources, time
- For each one establish a good definition of an example decision and it’s owner.
- Each team member then puts themselves in the position of the owner (this is even better if the owner is in the room and is included) and privately follows the decision tree answering truly honestly
- Each team member then shows the resulting decision making option they’ve chosen
- Consensus or conflict is then discussed. Outliers on the scale should be discussed first.
This simple exercise will get the team to understand each others motivations and approaches to problem solving as well as open their eyes to some of this stuff from a different perspective. Finally unknown conflicts in terms of decision making way well emerge where people had different assumptions of group input vs. directive management allowing them to be solved practically before a real personal conflict occurs.
To see how I apply this stuff to self-organising teams see: What does a self-organising team really mean? Organisation!
Finally…
Try applying this approach to personal relationships and decision making, not all of it will apply but it’s interesting to see where you act collaboratively with a partner vs. when you make an autocratic decision (or expect them to) and what the justification is for such a perspective. A misunderstanding between assumed decision making models is one of the underlying causes of many personal conflict situations, you can avoid these by understanding them.
Spend some time understanding how and why you make decisions both personally and professionally and you’ll reap some great rewards.
This blog is part of a series on Holistic Communication: The linguistics of business change. Introduction, ethics and table of contents is all in the first post.
References
- Vroom, Victor H.; Yetton, Phillip W. (1973). Leadership and Decision-Making. Pittsburgh: University of Pittsburgh Press.
- Vroom, Victor H.; Jago, Arthur G. (1988). The New Leadership: Managing Participation in Organizations. Englewood Cliffs, NJ: Prentice-Hall.
A model for making group decisions
I’ve been working with some interesting group structures recently which got me thinking about decision making dynamics. I’ve been using a decision making framework for a while and intend to apply it to some different shaped groups so thought I’d share it.
There are a number of irrational ways of making decisions such as divination, looking at guts or rolling a dice but generally in business we like to make logical rational decisions. However often we don’t really think about how we make decisions, or more importantly how we should make decisions.
The techniques described in this blog are applicable to any situation where a group needs to make a decision, however the language is focussed on business groups/teams.
This blog is in two parts, the first is pretty simple and obvious and covers basic decision making, but it’ll lead on to the more interesting second part about different ways of reaching agreement and when to use each method.
How decisions are made
Rational decisions are made via a fairly simple process, at least in my mind:
 Rational Decision Making |
1. Objective: We start with an objective or goal, it needs to be clearly understood and have an owner/leader2. Alternative options: we create some alternate solutions.
Idea generation is an interesting topic by itself and lots has been written, but that’s not the topic of this blog. 3. Tentative Solution: We analyse the alternates against some criteria and then choose one to be our tentative solution Again lots of ways to do this depending on your problem domain 4. Deeper Analysis: We perform deeper analysis on the tentative solution to try and ensure it’ll meet the objective 5. Agreement: We get the group to decide on the solution (see part 2) 6. Communication: We communicate the decision There’s no point making a decision unless it’s communicated! |
Person icon I used in the decision making diagram used as per attributable license from http://www.elegantthemes.com/
Different Types of decisions
The first thing to consider is that different types of decisions may need different types of decision making processes, specifically around gaining agreement and analysing options. This seems obvious to me, and is normally obvious to anyone I talk to about it, however when business change people are talking about creating/redefining a group to make some decisions they usually say how they’ll make decisions without differentiating between the types. Unfortunately common sense is often left out of planning.
A decision can’t be made unless it’s understood. To understand why a decision needs making someone, or the representative of some people, needs to have an objective which requires a decision by the group. Again, this seems rather obvious but if you consider it the other way around it means that a group can’t make a decision unless it know who wants the decision and why they want it. Apply that to many traditional management groups/boards, CCBs, business change projects, process improvement groups etc. and you’ll find it doesn’t always hold. In these cases groups are being dishonest and self-serving.
I don’t mean that you can’t make a decision without the customer present (models for that follow next) but that you can’t make a good decision if you don’t know who the customer is and what their objective is. In the case of speculative development for a market this might seem impossible but that’s what market research is for. So, given that we know how decisions are made, and that we should consider different ways to achieve agreement for different types of decision how can we actually get agreement?
See part 2 for How to reach agreement in a group – autocracy vs. democracy
This blog is part of a series on Holistic Communication: The linguistics of business change. Introduction, ethics and table of contents is all in the first post.
Definition of done
Teams are sometimes split into smaller sub-teams. Almost always they are part of a bigger team, which in turn is part of a bigger team and so on and so forth until we reach the organisational entity. Don’t get me started on B2B teams. Imagine an onion, with lots of layers. A weird mutated onion with multiple cores and overlapping onions. One that has tendrils sticking out into other similarly mutated onions like a big brain… Ok metaphor taken too far. Back to the topic…
The point is that everything team is connected to everything else. At every boundary it’s necessary to have a “Definition of Done” so that teams and their stakeholders understand what they’re getting. Any good or otherwise set of software development practices (like Scrum, RUP, waterfall or whatever you’re using) should help the team define the levels of their Definition of Done. I’m not using a specific example so I’ll just make some up off the top of my head to illustrate the point:
- Fully documented, discs printed, shrink wrapped and online distribution site fully functional, support systems in place ready for users.
- Beta test program
- Acceptance Tested
- Integration Tested
- System Tested
- Unit Tested
- Code Built
- Code written
- Code designed
- Idea thought of
- Requirement captured
Given some levels like this we need to ask the question where do we draw the line? As an R&D team, as a component team, as an integration team or business team I’d expect different answers. And for those answers to vary according to points in the lifecycle and relationships between teams and stakeholders.
The point is that these different levels of done take different amounts of work and are intended to differentiate different levels of quality (although I’ve used a fair few public release versions of software that don’t appear to have even been unit tested!). Without working out what the definition of done is between teams or parts of a team no one can estimate, plan or deliver any work.
If you have problems with multiple teams working together in your organisation, a lack of understanding of the Definition of Done is one of the common causes and is relatively easy to fix.
Use Case vs. User Story
Use Cases are too big to fit into a sprint/iteration! User Stories are so fine grained there’s too many too keep track of! Where’s the big picture? How to we define releases? Argh!!! I don’t know which to use!
Personally I tend to use both. I don’t think there’s any conflict between Use Cases and User Stories, in fact they’re rather complementary. Here’s how and why….
A few years ago I delivered a presentation at an agile software development conference entitled “Do requirements still matter?”. The short answer was “yes”. Most descriptions of agile iteration start with having a backlog, prioritising etc. etc. But where does the backlog come from? My answer: the Backlog Fairy; obviously.
Of course requirements still matter, they’re how we get a common understanding between customers and development teams of what we need to do. They’re the units by which we can incrementally build systems. In the rush to adopt agile processes a lot of teams have forgotten that they still need to start up with some lightweight analysis of scope and risk.
One of the things Use Cases are great at is capturing the big picture, showing a simple diagram of stick people and blobs that people who’ve never heard of UML can happily discuss in terms of what big bits of functionality are in and out of the system and who’s going to do them. The humble Use Case diagram. Not so easy to do that with a massive list of small stories.
However Use Cases can often be a bit too big to fit into a single development cycle but since they’re made up of a lot of different scenarios they’re easy to slice up into smaller bits. This is where I often use stories.
The advantage of Use Cases defining the scope is they’re nice chunky things to estimate and prioritise in a first pass. Of course as we get going we’ll estimate and prioritise stories for sprints/iterations but Use Cases help us prioritise requirements into Releases.
So I use a Use Case diagram to describe the high level scope of the thing to be developed. This normally takes about 5minutes to sketch. Then I use the Use Cases identified by the diagram (not documents, just the ovals on the diagram) to focus discussion on deriving stories (or use case slices) for development and testing within a sprint/iteration.
Sometimes stories turn up that don’t really fit into the early Use Case model, this is a rather good thing as it lets us challenge the understanding of scope. Does the story not fit in because it’s not really in line with the customers priorities and needs, it’s a cross-cutting or architectural concern or because we’ve missed some important part of the scope? All are important things to understand.
Mike MacDonagh's Blog 