Mike MacDonagh's Blog

Somewhere in the overlap between software development, process improvement and psychology

Tag Archives: agile@scale

No more Project Managers, bring in the Movie Producers

I was reading some course material recently that was trying to teach people something to do with software development and it was using the same old tired “ATM machine” example. I’ve worked with hundreds of projects, many in the finance sector and none of them are anything like an ATM machine.  One of the reasons that example is soooo tired is that it’s describing commoditised software development, it’s something I’d expect to buy off a shelf or go to a specialised vendor for. It’s not something I’d put my team of uber |33t  haxorz on.

Developing software products is a balance between a creative and scientific pursuit, so it’s a little hard to describe, especially when that software can range from a tiny smartphone app, to a website, to router firmware, to an enterprise hr system (urgh!), to a big data processing system of systems etc. You get the gist.

The things these types of system have in common with each other is how different they are. And yet the traditional approach to managing these diverse kinds of work has been classical project management with a one size fits all process (I’m looking at Prince2, RUP, Iterative, Agile etc.) or worse hiring a large company full of body-shop project managers. For me this is one of the root causes behind large scale IT disasters.

I once had a discussion with the leader of a PMO in a large organisation of thousands of people about the nature of Project Management for software and he assured me that “someone who knows how to manage a bit of work can manage software, it’s just a different type of work” and indeed I should “stop thinking of software as special”. I’ve seen this attitude resonate in many organisations and is to me best summed up by the same Head of PMOs statement “from a project management point of view building software is the same as building a bridge”.

Now then. I know a little about software development, but not that much about bridge building (if you exclude my illustrious Lego engineer career when I was 7). If I managed the building of a bridge next door to one build by someone with a track record of bridge building who’s bridge would you drive your family on? Not mine if you’ve got any sense.

There are so many problems in the software development industry caused by people not understanding it and trying to apply their outdated knowledge at best or badly aligned metaphors at worst. When people who do understand it start to get the point across (e.g. the early agile movement) they soon start selling certifications and losing their grip.

Many of our “best” metaphors come from manufacturing lines where the same thing is made over and over again. That’s nothing like software.

To me a software project is more similar to making a movie than anything else:

  • It’s a unique one off product (or a remake of a previous version) with new parts working together in new ways
  • Once made we’ll often duplicate and digitally distribute
  • It can cost a lot of money because it can be very complex and needs lots of specialist skills
  • You need a high level plan for how you’re going to make it
  • There’s no guarantee on return
  • What’s written down originally bares little resemblance to the finished product
  • We know when it’s good or bad but it’s hard to objectively quantify
  • There’s lots of different types that need different collections of specialist skills
  • Both involve people and time and so are subject to change, they have to be adaptive
  • You can tell when the people making it had fun
  • It’s not feasible that any team member can fit in any role
  • There’s almost always going to be some quality problems
  • You wouldn’t get a movie maker to build your bridge
  • You wouldn’t get a bridge builder to make your movie
  • You don’t make a movie faster by telling the actors to act faster

So, I don’t want a Project Managers that know how to work a gannt chart. I want movie producers that know how to work with a team to get the best out of them both technically and artistically.

 

How to make real progress in a software project

I’ve not been blogging much this year because I’ve been crazy busy writing software (more on that soon I hope). As you might expect I’ve changed my ways of working more often than process improvement consultants change the name of the process they’re selling but one key thing always seems to drop out whether I’m working on my own, with a team, a team of teams or a globally distributed team.

Do the top thing on the list, whatever it is.

Don’t do anything else until it’s done.

Or in process speak: “Reduce the WIP and clarify the Definition of Done“.

Too often I fall into the trap of doing the most interesting thing from the list, which normally means the most complex. And far too often I get carried away with the next interesting thing before finishing something. Both of these lead me to have far too many things in various states of progress all interfering with each other and me with no idea of how well I’m doing. And that’s assuming I actually know what “finished” means.

Software is complex enough without us making it harder with processes, remember the KISS principle?

Resonating social patterns with project processes

People are social complex agents, organising people is a bit like herding cats, however when people are working together collaborating in teams they can achieve amazing things. So why is it that some approaches and teams structures work and others seem to cause problems?

I’ve been thinking about things like nudge theory, servant-leadership, agility in software development, lean business, social business, serious gaming and agile at scale. Most people tend to think that these are all (or mostly) good things, they’re often desired bottom-up in businesses and support “faster, cheaper, better, happier” agendas. All good things that tend to be desirable at every level of a business and yet a lot of my work is about helping individuals, teams and businesses towards this simple agenda using these kind of things because although the goal is simple and the change is desirable, actually doing the change isn’t easy.

All of these things seem to “feel right” to people and are generally desirable and yet they are often at odds with traditional management techniques which tend to compartmentalise people and decompose everything into linear hierarchies.

So why is it that treating people like complex social creatures works better than treating them as simple functional unit? Er… because that’s what they are, people. I’d love to know why people think the opposite can ever work?

I think that the reason that the list of things up there is generally so successful at working towards faster, cheaper, better, happier business is that they are socially resonant. That is, ways of thinking about work such as agile software development are congruent with normal human social behaviour, that’s why they work.

Nudge theory acknowledges that people are lazy and don’t do what you tell them to. Delayed gratification doesn’t motivate most people so put things you want people to do in their way and gently nag them about it. Make it easier to do the right thing, make the wrong thing harder and more formal. The rise of the adult playground is a great example of this. People don’t want to be told what to do, they want their ability and contribution to be valued. Servant-leaders are ideally placed to nudge people, which they can do based on their personal social relationships.

One of the reasons that the agile movement has taken off as well as it has is because it treats people like people, in fact that’s in the agile manifesto! By aligning ways of working to normal human behaviour you are enabling your team to get on and do things intuitively, normally and comfortably.

Photo of east gate of Roman Forum

Photo by Mykola Swarnyk

I’ve been applying this kind of thinking to large scale project structures and that’s led to the Project Forum practice (think Roman Forum rather than phpBB!) which I’ve described as a “middle-out” management structure as it’s not bottom-up or top-down. Instead it’s more like a tribal council bringing together the leaders of other groups to an area where they can all have their voices heard. It’s democratic and social, it doesn’t pretend there isn’t any conflict instead it provides a vehicle to resolve that conflict. This structure resonates with democratic political structures from the tribal council all the way to parliamentary democracy.

In this model, the Project Manager has a pressure from the business to deliver and he gets to impress this upon the other members. Customers with a pressure for quality or short-term goals get to understand why their concerns need balancing with scope and resources. Contributing teams get to have their agendas and issues collaborated on by the wider group and can manage supply and demand of their resources. Wherever there is conflict the way to resolve it is through open honest communication, the Project Forum is that vehicle, providing a sort of open parliament. Yeah, I know it’s not a great name but I’m not good at naming things.

Most cultures have evolved away from autocratic dictators towards representative democracy in one form or another because that’s the way people want to collaborate socially. So why not apply the same model to large projects? Thousands of years of history already tells us it works.


This blog is part of a series on Holistic Communication: The linguistics of business change. Introduction, ethics and table of contents is all in the first post.

Scaled Agility: The Project Forum

Name: Project Forum (Middle-out management structure) – Agile at Scale practice

When it might be appropriate

  • In situations where multiple competing stakeholder groups with different agendas are required to work together
  • In situations where multiple product groups need to collaborate on a bigger outcome
  • Where there is a conflict in direction, resource management/ownership or scope between collaborating groups
  • System of systems development

What is it?

The Project Forum is an application of agile philosophy to large project structures. Rather than impose a hierarchy of decision making from the Project Manager downwards the Project Forum is a virtual team in the middle of all stakeholders.

The Project Forum is a self-organising democratic group that balances competing voices and concerns, owns high level scope and architecture, runs the high level release train and performs integration activities for the product.

Use of the Project Forum practice does not prevent any communication directly between contributing groups it only provides a vehicle for that conversation when it’s relevant for the wider project.

From Traditional to Agile at ScaleThe Project Forum practice is an example of Agile at Scale combining social business practices, technical software practices and ways of working to make a simple way of doing big complicated bits of work.

Read more of this post

Simple software project measures

I’m not a big fan of metrics, measures, charts, reporting and data collection. I’m not terribly impressed by dashboards with 20 little graphs on showing loads of detailed information. When I’m involved in projects I want to know 3 simple things:

  • How quick are we doing stuff?
  • Are we on track or not?
  • Is the stuff good enough quality?

There can be some deep science behind the answers to those questions but at the surface that’s all I want to see.

Organisations need to know that teams are delivering quality products at the right pace to fit the business need. To achieve this goal teams need to be able to demonstrate that their product is of sufficient quality and that they can commit to delivering the required scope within the business time scales. If the project goal may not be achieved then the business or the team need to change something (such as scope, resources or time scales). This feedback mechanism and the open transparent communication of this knowledge is key to the success of agile delivery.

The goal of delivering quality products at the right pace can be measured in many complex ways however, when designing the Project Forum agile at scale practice we looked at just 3 measures. In fact I should probably call them 2.5 measures as the throughput/release burnup can be considered mutually exclusive (if you’re continuous flow or iterative). The most important measure is people’s opinions when you go and talk to your team.

Simple Measures Dashboard

Note: in the measures section I often refer to “requirements” as a simple number, this could be a count, a normalised count, magnitude, points, etc. it doesn’t matter what’s used so long as it’s consistent.

Read more of this post

What does “Agile at Scale” mean?

There’s a lot of talk in the process improvement industry about the meaning of “agile at scale”, devops, lean and agile for a number of reasons. One reason that I’ve seen in successful agile organisations is that as development maturity increases with true agile adoption bigger problems present themselves. This is the natural progression of the science of development, what used to be considered complex (like Object Orientation) is now normal, a commodity. Innovation in ways of working is happening at the organisational, cross-team, cross-product level.

For me agile at scale (I’ve got fed up of the quotes already) means a couple of different things:

  • Repeating agile successes embodied in a team across an organisation (scaling out?)
  • Applying agile thinking to cross-product projects
  • Applying agile and lean thinking to development organisations
  • Applying agile and lean thinking to high assurance environments like medical, security, financial, safety critical, audited, regulated businesses.

Agile and lean? Yep, both with lower case letters. I’m not particularly interested in ideological approaches to software development, I believe strongly in taking the best bits of whatever processes, techniques, practices etc. you find externally, mixing them up with internal practices and ways of doing things to develop simple, pragmatic approaches to ways of working. Both agile and lean schools of thought promote minimising unnecessary work, shorter delivery cycles and higher quality, continuously learning lessons and empirical decision making.

The agile manifesto gave us:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on
the right, we value the items on the left more.

Some great practices have evolved for applying agile and lean thinking like scrum, kanban etc. However all of the complex organisations I’ve worked with have found that there’s still space for more thinking in terms of how to run a software business, how to deal with big complex system-of-system problems, multiple competing stakeholder sets, programme and portfolio management etc. Not surprising really because the agile movement wasn’t about trying to do any of that stuff.

However organisations who are successful with agile transformations want to apply the successful open and honest philosophy behind the agile manifesto to other parts of their business as well as bigger and bigger projects and programmes because the results of “doing agile” (I promise I’ll stop with the quotes soon!) are so attractive when it’s done well, namely:

  • Shorter delivery cycles, higher quality
  • Deep engagement between customers and development teams leading to respect, collaboration and better morale
  • Quick identification of when things are starting to go wrong

Consider the following model, not uncommon amongst large organisations:

Diagram of nominal large inter-dependant organistion structure

This represents a typical software department or vertical section of a software department with a portfolio that provides the funding for work. Big portfolio’s are normally broken down into a number of programmes which in turn may levy high level requirements onto organisations (organisational sub-divisions that own multiple products) which may affect both legacy and new product development. Often within a vertical section of a business there will be many cross-dependencies in terms of requirements, technical dependencies etc. For many large businesses this picture is overly simplistic, indeed I’ve not included things like projects, component teams and a variety of business organisation constructs like product centres, feature teams etc.  So how do you apply agile and lean philosophy to this lot and more?

You can’t simply repeat the same practices recursively throughout an organisation to deal with larger scale complexity.  Imagine a chain of scrum-of-scrums, daily stand-ups at every level (at the same time, or staggered to allow representation up the chain?), sprint plans at programme level etc. What about if the business is regulated, audited, security focussed, high risk financial, safety critical, etc.

Ok, so what’s agile at scale then?

Agility at Scale is applying the spirit of agility and lean thinking if not the letter to these bigger problems. It’s about:

  • Valuing individuals and interactions, encouraging collaboration, reducing layers of communication over processes, tools and hierarchy
  • Valuing working software in the form of quality releases from short development cycles over comprehensive documentation, business analysis, enterprise architecture documentation
  • Valuing customer, business, developer and operations (see DevOps) collaboration over contract negotiation
  • Valuing good governance, transparency and honesty in progress, plans, costs and impediments over regular reporting
  • Valuing responding to change over following a plan at all levels of the business

Agility at scale is focussed simply on reducing unnecessary bureaucracy, reducing time to market and improving value.

So how do you achieve it?

The application of:

Of course each of those (and more!) is a complex can of worms in itself. A lot of these higher scale practices are only just emerging from high maturity complex (post-)agile organisations but in time more of those things will turn into links.

A good example of  “Agile at Scale” in action is the Project Forum practice

As always this blog is  a stream of consciousness so please, let me know your opinion?

Follow

Get every new post delivered to your Inbox.

Join 345 other followers

%d bloggers like this: