Mike MacDonagh's Blog

Somewhere in the overlap between software development, process improvement and psychology

Tag Archives: process

No more Project Managers, bring in the Movie Producers

I was reading some course material recently that was trying to teach people something to do with software development and it was using the same old tired “ATM machine” example. I’ve worked with hundreds of projects, many in the finance sector and none of them are anything like an ATM machine.  One of the reasons that example is soooo tired is that it’s describing commoditised software development, it’s something I’d expect to buy off a shelf or go to a specialised vendor for. It’s not something I’d put my team of uber |33t  haxorz on.

Developing software products is a balance between a creative and scientific pursuit, so it’s a little hard to describe, especially when that software can range from a tiny smartphone app, to a website, to router firmware, to an enterprise hr system (urgh!), to a big data processing system of systems etc. You get the gist.

The things these types of system have in common with each other is how different they are. And yet the traditional approach to managing these diverse kinds of work has been classical project management with a one size fits all process (I’m looking at Prince2, RUP, Iterative, Agile etc.) or worse hiring a large company full of body-shop project managers. For me this is one of the root causes behind large scale IT disasters.

I once had a discussion with the leader of a PMO in a large organisation of thousands of people about the nature of Project Management for software and he assured me that “someone who knows how to manage a bit of work can manage software, it’s just a different type of work” and indeed I should “stop thinking of software as special”. I’ve seen this attitude resonate in many organisations and is to me best summed up by the same Head of PMOs statement “from a project management point of view building software is the same as building a bridge”.

Now then. I know a little about software development, but not that much about bridge building (if you exclude my illustrious Lego engineer career when I was 7). If I managed the building of a bridge next door to one build by someone with a track record of bridge building who’s bridge would you drive your family on? Not mine if you’ve got any sense.

There are so many problems in the software development industry caused by people not understanding it and trying to apply their outdated knowledge at best or badly aligned metaphors at worst. When people who do understand it start to get the point across (e.g. the early agile movement) they soon start selling certifications and losing their grip.

Many of our “best” metaphors come from manufacturing lines where the same thing is made over and over again. That’s nothing like software.

To me a software project is more similar to making a movie than anything else:

  • It’s a unique one off product (or a remake of a previous version) with new parts working together in new ways
  • Once made we’ll often duplicate and digitally distribute
  • It can cost a lot of money because it can be very complex and needs lots of specialist skills
  • You need a high level plan for how you’re going to make it
  • There’s no guarantee on return
  • What’s written down originally bares little resemblance to the finished product
  • We know when it’s good or bad but it’s hard to objectively quantify
  • There’s lots of different types that need different collections of specialist skills
  • Both involve people and time and so are subject to change, they have to be adaptive
  • You can tell when the people making it had fun
  • It’s not feasible that any team member can fit in any role
  • There’s almost always going to be some quality problems
  • You wouldn’t get a movie maker to build your bridge
  • You wouldn’t get a bridge builder to make your movie
  • You don’t make a movie faster by telling the actors to act faster

So, I don’t want a Project Managers that know how to work a gannt chart. I want movie producers that know how to work with a team to get the best out of them both technically and artistically.

 

Agile contracts? What about Weighted Fixed Price?

MichaelangeloAgile contracts are tricky. In fact Software Development contracts are tricky. Software development is a non-deterministic creative endeavour, you’d be better off trying to nail down the acceptance criteria for one of Michelangelo‘s sculptures than a typical software project. Software is complex.

A few years ago I was involved in putting together a contract model called “Weighted Fixed Price” and recently it occurred to me that I’ve never blogged about it, so I’ve dragged myself away from the source to do so.

What’s wrong with the normal models?

So, fixed price is bad because it only works when you can really accurately define the requirements, acceptance criteria and the potential risk exposure is tiny. Otherwise you get schedule and cost over-runs, formal change control etc. etc. Of course supplier’s always build in their risk buffer into the fixed price so it tends to cost a little more than honest T&M would. Software work is rarely this predictable which is why there are so many high profile failures.

Time and Materials is bad because it’s hard to manage across commercial boundaries and customers can feel that they’re paying for nothing, endlessly. Even defining waypoints is difficult due to changes in requirements and over time. Unscrupulous suppliers could just sit back taking the money, delivering as little as possible to avoid gross negligence but nothing more. Scrupulous suppliers could be working hard but hit by the many and varied risks arising from such complexity and yet perceived by their customers as the unscrupulous.

Of course there’s variations like Capped T&M, Phased Fixed, Phased Capped T&M etc. but they all start getting a bit complicated and hard to manage, not to mention open to gaming.

Then there’s models like pay-per-point and other “incentive” schemes that raise over-complication, missing the point and gamability to an art form. Just say “no”.

Competition

The real incentive to suppliers to do a good job is follow on work. That means that as a customer you need your suppliers to be in a competitive market, being locked in to individual suppliers leads to captive market behaviours.

Weighted Fixed Price?

In an attempt to put something in the middle between Fixed Price for low risk exposure and T&M/internal for high exposure I put together “Weighted Fixed Price”. The idea here is to to basically use a Fixed Price model but have suppliers weighted by previous performance affecting whether they get the business or not.

So if a supplier 1 delivers 10% late on Project A then when we’re looking to place Project B for 500k and we’re looking at each supplier’s estimates then we’ll add 10% to supplier 1′s estimate.  That means all over things being equal we’ll choose the other supplier’s who typically come in closest to, or even under, their previous estimates. Weighting is transparent to each supplier but not across suppliers.

This provides the following pressures:

  • Suppliers don’t want to underestimate, they’ll come in over their initial estimates and adversely affect their weighting and therefore their chance for follow on work
  • Suppliers don’t want to overestimate to artificially lower their weighting or build in risk buffers, they won’t get the work because they’ll cost too much
  • Suppliers don’t want to take on complex, high risk work because they can’t predict the effect on their weighting (more below)

Out-sourcing doesn’t transfer risks to suppliers, because the customer is still the one needing the business benefit and paying for it, especially when things go wrong. The Weighted Fixed Price model is intended to genuinely transfer the complexity risk to suppliers, if they’re willing to take it.

What happens when the supplier finishes early?

They deliver early, watch their weighting affect their potential for winning future work increase massively and enjoy their fixed price money meaning that they’ve increased their profit margin on this job.

What happens when they deliver late?

They deliver late, their weighting increases and they’re less likely to win against other suppliers next time. However, if they could predict that they’re going to deliver late early in the project then they could renegotiate with the customer in light of what everyone’s learned about the work.

What happens when suppliers don’t want to bid for work because it’s too risky/complex?

Excellent, this means that they’re honestly telling us that the work is too risky to predict. This means we shouldn’t be out-sourcing it in the first place, time to stand up an internal team or supplement with some time-hire contracts if we need extra capacity.

Conclusion

The Weighted Fixed Price model is an evidence based contract model can be used in competitive supplier markets for reasonably well understood work with low risk exposure. It incentivises suppliers to estimate accurately and deliver to their estimates as that affects their opportunities for follow on work. It incentives customers not to out-source overly complex risky work.

It’s not perfect because we’re still trying to contractually define how a sculpture should make us feel but it’s something you can add to the mix of contract models you use.

How to make real progress in a software project

I’ve not been blogging much this year because I’ve been crazy busy writing software (more on that soon I hope). As you might expect I’ve changed my ways of working more often than process improvement consultants change the name of the process they’re selling but one key thing always seems to drop out whether I’m working on my own, with a team, a team of teams or a globally distributed team.

Do the top thing on the list, whatever it is.

Don’t do anything else until it’s done.

Or in process speak: “Reduce the WIP and clarify the Definition of Done“.

Too often I fall into the trap of doing the most interesting thing from the list, which normally means the most complex. And far too often I get carried away with the next interesting thing before finishing something. Both of these lead me to have far too many things in various states of progress all interfering with each other and me with no idea of how well I’m doing. And that’s assuming I actually know what “finished” means.

Software is complex enough without us making it harder with processes, remember the KISS principle?

Reduce waste: Visualise the value stream

A big thing in software process improvement (urgh) is reducing waste.  A great idea, but how do you identify waste?

One powerful method that I use is value stream visualisation. There’s a number of ways of getting information about the value stream ranging from physically walking the chain to simply asking people.

Sounds simple doesn’t it? But one of the best sources of information about wasteful processes and ways of working in a team is simply asking the team what they feel they shouldn’t be spending time on. By freeing up people to actually do their job they can reduce waste and improve productivity. One method I use is to ask each team member to identify a list of n items that they have to spend time on that they think are either entirely or partially wasteful. Basically a list of their impediments. I then anonymize the list look for commonality and then the top few are a reasonable target for improvement.

Alternatively actually walking the path of a piece of work through an organisation can really open the eyes to the sheer number of people and unnecessary waiting involved in some wasteful processes…

Anyway, having got some data I find it useful to visualise it in terms of waiting time and doing time. In Lean-speak cycle time is the time it takes to actually do something, lead time is the time it takes from the request being made to it finally being delivered. In many tasks the lead time far outstrips the cycle time, visualising it can make that very clear.

Low Lead time For example, a task with has a lead time of 5 days might actually be made up of 2 days waiting time before it actually gets picked up and the cycle time of 3 days starts.

Large lead timeIf you look at all of the tasks in your value chain you might find that many of them look more like this, with very large wasteful waiting times. There are many reasons why there might be a large waiting time, typically the team doing the tasks is over-subscribed, or a team might be protecting it’s SLAs, KPIs or other TLAs. These problems can have significant effects when looking at the system as a whole.

Even more interesting is looking at how these tasks might fit together in a value stream. If we imagine a business function that involves a request moving through a bunch of tasks across a number of teams that have different waiting and doing times a simple sequence of tasks, with a little buffering between them from the Project Manager to allow for schedule variance naturally, might end up looking like this:

 sequential_value_chain_arrows

Here we have 26 days of actual work spread over almost 60 days (12 weeks) of elapsed time = 40% waste. Even if the person planning the work is willing to accept some risk and try to optimise their workflow a bit without improving the waste inherent in the tasks involved there’s still a lot of waste in the system.

parallel_value_chain_arrows

By visualising the value stream in this way we can see straight away (apologies to red/green colour blind folks) that there’s a lot of red, a lot of waste. In many cases planners aren’t willing to accept the risks inherent in overlapping activities as shown here, or aren’t even aware that they can leading to the more sequential path shown above. The result is a minimum time that it takes a request to get done, based on the impedence of the current value chain of, in this case, 38 days before we even start thinking about actually doing any work.

Surely that’s not right.

Intentional vs. Emergent Architecture

I’ve been thinking about architecture a lot recently but one thing that I often discuss but have never blogged about for some odd reason is intentional vs. emergent software architecture. Some old fashioned software methods such as waterfall led people into doing a lot of up front architecture work, they analysed and designed away for ages producing huge reams of UML and documentation that no one could ever squeeze into their heads if they had the patience to read it. This is an example of an intentional architecture – the architecture was intended, planned and deliberate.

Lots of folks hated that way of doing things as it meant people were writing docs and drawing diagrams instead of making working software, not to mention an all to frequent tendency to over-engineer architecture past the point of usefulness. This led to some people saying that we’re better off not trying to do any architecture and just letting it emerge from the work we do developing small little customer focused requirements (like user stories or similar).  Ok, so there’d be some rework along the way as we encounter a deeper understanding of the system and refactor our emergent architecture but it’d still be better than the old way of doing large upfront architecture.

So, there seem to be two opposed viewpoints: intentional architecture is best, emergent architecture is best.

For me, neither is true. I’ve seen some really terrible examples of badly over-engineered architectures that crippled a project and projects that never got past their over-reaching architectural analysis. Equally I’ve seen products with emergent architecture that had to be entirely re-architected as time went pay because their emergent architecture was so wrong it was comical (imagine a software management tool that only supports a single project, and that concept being deeply embedded in the architecture).

There’s a scale with intentional architecture on one side and emergent architecture on the other.

Intentional vs. Emergent ArchitectureVarious factors might push us one way or another… The second I listed on the right is interesting as if you’ve got a well known technology and problem domain you can get away with emergent architecture, but similarly if you have a totally unknown technology and problem domain it can be very effective to evolve towards a solution and architecture rather than crystal ball gaze by creating a (probably wrong) intentional architecture.

Which rather sums up the point I’m trying to make. The purpose of architecture is to shape the solution and address technical risks. Solving the big problems, creating common ways of doing something (lightweight architectural mechanisms) are all good architectural goals but only if we’re sure of the solution. If we’re not we’re better off evolving an emergent architecture, at least initially.

I think that the extremes at either end of the scale, as with most extremes, are a bad idea at best and impossible at worst. If you gather a group of people together and tell them to create a web app given a backlog but they’re not allowed to think or communicate about the architecture up front then you’ll find they all start dividing the problem in different ways and writing in different languages for different server and client frameworks. Hardly a good idea. Of course on the other end of the scale, believing that we can foresee all of the technical issues, all of the technology and requirements changes that might happen is as likely as a 12 month project plan Gantt chart being correct after a few levels of cumulative error margin have been combined.

For more on architecture see:

Quality Confidence: a lead measure for software quality

For a long time I’ve wanted to be able to express the quality of my current software release in a simple intuitive way. I don’t want a page full of graphs and charts I just want a simple visualisation that works at every level of requirements to verification (up and down a decomposition/recomposition stack if you’ve got such a beasty). My answer to that is Quality Confidence.

What it is

RAG GaugeQC combines a number of standard questions together in a single simple measure of the current quality of the release, so instead of going to each team/test manager/project manager and asking them the same questions and trying to balance the answers in my head I can get a simple measure that I can use to quantitively determine whether my teams are meeting our required “level of done“. The QC measure combines:

  • how much test coverage have we got?
  • what’s the current pass rate like?
  • how stable are the test results?

We can represent QC as a single value or show it changing over time as shown below.

How to calculate it

Quality Confidence is 100% if all of the in scope requirements have got test coverage and all of those tests have passed for the last few test runs.

To calculate QC we track the requirements in a project and when they’re planned for/delivered. This is to limit the QC to only take into account delivered requirements. There’s no point in saying we’ve only got 10% quality of the current release because it only passes some of the tests because the rest having been delivered yet.

We also track all of the tests related to each requirement, and their results for each test run. We need to assert when a requirement has “enough coverage” so we know whether to include it or not – the reason for this is that if I say a requirement has been delivered but doesn’t yet have enough test coverage then even if all of it’s testing has passed and been stable then I don’t want it adding to the 100% of potential quality confidence. The assertion that coverage isn’t enough means that we aren’t confident in the quality of that requirement.

So 100% quality for a single requirement that’s in scope for the current release is when all the tests for that requirement have been run and passed (not just this time but for the last few runs) and that the requirement has enough coverage. For multiple requirements we simply average (or maybe weighted average) the results across the requirements set.

If we don’t run all the tests each during each test run then we can interpolate the quality for each requirement but I suggest decreasing the confidence for a requirement (maybe by 0.8) for each missing run. After all just because a test passed previously doesn’t mean it’s going to still pass now. We also decrease the influence of each test run on the QC of a requirement based on it’s age so that if 5 tests ago the test failed it has less impact on the QC that the most recent test run. Past 5 or so (depending on test cycle frequency) test runs we decrease the influence to zero. More info on calculation here.

So… how much coverage is enough?

Enough coverage for a requirement is an interesting question… For some it’s when they’ve covered enough lines of code, for others the cyclomatic complexity has an impact, or the number of paths through the requirements/scenarios/stories/use cases etc. For me, a requirement has enough test coverage when we feel we’ve covered the quality risks. I focus my automated testing on basic and normal flows and my human testing on the fringe cases. Either way, you need to make the decision of when enough is enough for your project.

To help calibrate this you can correlate the QC with the lag measure of escaped defects.

Measurement driven behaviour

Quality ConfidenceThe QC measure is quite intuitive and tends to be congruent with people’s gut feel of how the project/release is going, especially when shown over time, however there’s simply no substitute for going and talking to people. QC is a useful indicator but not something that you can use in favour of real communication and interaction.

The measurement driven behaviour for QC is interesting as you can only calculate it if you track requirements (of some form) related to tests and their results. You can push it up by adding more tests and testing more often :) Or by asserting you have enough coverage when you don’t :( However correlation to escaped defects would highlight that false assertion.

If you’ve got a requirements stack ranging from some form of high level requirements with acceptance tests to low level stories and system tests you can implement the QC measure at each level and even use it as a quality gate prior to allowing entry to a team of teams release train.

Unfortunately, because me and my mate Ray came up with this in a pub there aren’t any tools (other than an Excel spreadsheet) that automatically calculate QC yet. If you’re a tool writer and would like then please do, I’ll send you the maths!

I’m coming out as not-Agile and not post-Agile

Big A vs. Little a

Huh? What? I’ve written a fair bit on this blog about agile topics, but I always try to write about agility with a small “a”. I’m not really into Agile with a big “A” though – I’m not into doing things according to a set of rules and having arguments about whether I’m doing it right or not. I’m not anti-agile, but I’m increasingly anti-Agile.

To me, the ideological arguments, definitions of everything, frameworks, money-spinning certifications and money-spinning tooling are what’s wrong with doing “Agile”. Being empirical, reflecting and adapting, honestly communicating and putting people first as an approach is being “agile”.

I don’t really like the term “post-Agile” either though as it comes with a bunch of baggage and is easily misinterpreted – and I still see benefits in adopting agile practices. I don’t want to see another specific set of rules or a statement or beliefs with elitist signatures. For me what’s next after agile is about dropping the ideology in software process, destroying the ivory tower of trademarks, formal definitions, money spinning tools and money-spinning certification programmes.

We need to get rid of the League of Agile Methodology Experts and if anyone says “Ah, but this is different” when showing a website of process content then you have my permission to hit them with a stick.

So what does the future look like?

Software development is a complex social activity involving teams of people forming and self-organising to work together, sometimes in teams of teams which is even harder. As technology is increasingly abstracting up and raising in maturity so is the way that developers, managers and organisations think about software and doing software. I think the problem is getting increasingly social, and the solutions will start looking increasingly social using more “soft practices”.

Software process improvement agents/consultants/coaches/mentors (including myself) need to take a long hard look at themselves. Are they telling people how to do it right when they can’t even write HelloWorld in a modern language? I’ve said that to some acquaintances in the industry, generously qualifying “modern language” as something significantly commercially used in the last 10 years and seen them look quite offended at my insulting affront to their professional integrity. I’ll go out on a limb and say you can’t coach a software development team if you don’t know how to write software.

So… software process improvement?

TOverlapping concerns in process improvementhe world runs on software, it’s everywhere and it’s critical. Getting better at doing software, improving the software value chain, is a noble aim and will continue to be as it means getting better at doing our businesses and our lives.

For me process improvement (dislike that phrase as well) is going to be more about combining psychology based business change practices with the best bits of a wide variety of ways of working (agile, lean, Scrum, what you currently do, RUP, Kanban, various workflow patters etc.) with technical software development practices like continuous integration, continuous delivery.

We need to work together, not as “leaders and followers” or “consultants and clients” but as collaborative peers working together to apply specialist skills and experience so that we can all improve. Smart people should be treated as smart people, we have much to learn from them and should be thinking in terms of fellowship rather than leadership.

I’m calling this overlap “soft practices”* because the term is evocative of:

  • The practice of doing software
  • The practices involved in doing software
  • Being able to deal with people is sometimes called  having “soft skills
  • Soft power

What do you think about “post-Agile”?

* I’ve even set up a company called Soft Practice to do this stuff, that’s why I’ve not been blogging much recently, who knew there’d be so many forms to fill in!

Edit 14/8/13: Seems others are now talking about the same things: Ken Schwaber, Dave Anderson, Aterny

Process vs. Tools

What’s more important: Process or tools?

I’ve sat through numerous arguments with people trying to work out whether process or tools are more important to their software development effort. The arguments go along the lines of:

Process Geek: Tools are just the things you use to do the process, the value is in the process. Let’s do a process first roll out.

Tools Geek: Tools are the things people actually use. Process is academic and is constrained by the tool features anyway. Let’s do a tools based roll out.

Sadly both of these are real life quotes from people I’ve worked around in the past. I won’t embarrass them by naming them because they’re both really wrong. For me the real value is not in the tools or the process but in the people. Process and tools are something that you add if the people need them.

Minimal Tooling

For me the best planning tool is a whiteboard. The best architecture and design tool is also a whiteboard. I like to capture some info about what I’m doing and how close I am to achieving my goals, some really simple measures. The best tool for that is probably Excel, despite my love of Open Source. Other than that I need to manage some lists of things like requirements (stories/use cases whatever), defects and whatever other bits the team thinks are important. They might fit on a whiteboard but typically we want to have some attributes on them, be able to sort them and link them. The one downside of a white board is the lack of drag and drop. I’ve used a few different types of smart board and active touch screens but none of them have really given me the smooth experience I want.

The one bit of tooling I really need is my IDE + SCM + Continuous Integration.

Minimal Process

This is potentially a can of worms if I go into things like process kernels and ideological arguments about different agile approaches and practice collections. There’s no such thing as a process silver bullet so for me though it’s all about the team and being effective. Teams need to:

  • Have Clear Obligations
  • Have Clear Business Priorities
  • Seize their autonomy
  • Have the relevant technical skills and resource availability
  • Be active in actually doing their work (seems obvious but it’s something a lot of processes seem to hinder rather than help)
  • Demonstrate visible progress (via working software not a tracking gantt chart)
  • Continuously improve themselves (because no way of working is perfect)

This list based on the 7 Habits of Agile Teams by Roly Stimson

So why go more complicated?

Unfortunately life is rarely simple. Many teams work in complex environments where a number of factors might make them choose to need more than this kind of minimal set including things like physical distribution; audit, regulatory or security constraints; cross-team interaction in system of systems environments, more people etc.

When faced with increased complexity it might be appropriate to add more powerful tooling or development practices. The more people involved in a project the harder it is to herd all of those socially complex cats into working together.

This is why ALM suites exist ranging from plugging together some open source bits like git, SimpleGit, jenkins, agilefant, bugzilla etc. to commercial vendor products like IBM Rational RTC (which does agile planning, work item management, scm and build in one tool) and other IBM Rational Jazz tools – it’s worth noting that you can also get RTC for free for just 10 users or for academic/research use on jazzhub.

Alternatively there’s Microsoft TFS for source control, work item tracking etc. tying together existing Microsoft stuff like MS Office, Visual Studio and sharepoint if you’re a more Microsoft focussed development house.

Of course you can also mix and match this stuff to a greater or lesser degree with various integrations. No toolset is a silver bullet however and each have their own problems.

So what’s my point?

My point is that process and tools need to reinforce each other, that both should be selected based on the needs of the team and the environment the team has to operate in. In the process vs. tools argument both lose out to the people. People in a development organisation create value, they just use tools and process to do it.

Zen and the art of Enlightened Software Development

When I started doing software I was a simple developer interested in elegant code and shiny things. I worked in a small software house that taught me many bad practices in terms of configuration management, change control, estimation, management etc. It was an incredibly valuable experience for me in learning how not to do things and led me to strive for something better even though at the time I didn’t really know what it might be.

That led me through a path of process definition and documentation (RUP) and a rather limited following of an iterative lifecycle. I remember a conversation I once had with one of my project managers (when I was an architect) asking if we were going to do “Use Case Analysis” on a particular project or just skip straight into “Use Case Design”. The question puzzled me then because it felt like a trap and I didn’t really understand what I was being asked.

It puzzles me even more now as I know understand “analysis” to be a stop-and-think-for-a-moment activity and I’d always do a bit of analysis of my requirements, however these days it’s extremely unlikely that it’d be an analysis UML model, instead it might be a sketch, a conversation and a bit of thinking before another conversation.

So I moved from process prescription to process understanding, applying the spirit of doing things well rather than the letter of whatever current process law was in fashion. Following on from that I’m not really interested now in the details of what a book says someone’s role should be or how people should interact instead I think the real challenges in software development are social, not collections of technical practices (although there is  value in evolving better practices and tooling).

A colleague of mine commented recently that 20 years ago when he was doing software it was possible to understand everything from the metal all the way to the blinking lights on his bit of hardware. Software development in just 20 years has progressed incredibly and it’s just not that simple any more. We can understand the basics all the way through the stack but not all of the details. There are so many bits interacting that it’s just too complex. As a result the problems, technology and team working are all abstracting away from the tangible mechanistic past.

Developers are on a path from technical skill to mastery, as they begin to understand the kung fu of software engineering they can apply experience and deep understanding to solving the technical problems, doing away with formal process and just using the practices that they intuitively need, happily breaking the “rules” to get the job done in an efficient high quality way. I’m not entirely sure where brogrammers are on this evolutionary path, I’ll leave that up to you.

The problem, if it even is a problem, is that each person in an organisation is somewhere different along this path and even if they’re at the same point in the same dimension they might not be aligned in their interests and motivations. This makes team working amongst inherently complex social creatures a tricky proposition. The sweet spot is a team of fully enlightened software kung fu masters but that’s a really hard target to meet for a number of reasons. Consider the flower of team working evolution weirdness, which area is your team in?

One reason that this sweet spot is difficult to hit is because in any organisation half of the developers are below average in technical and social skills. Sounds horrible but is obviously true and the larger the organisation the more likely it is to be the industry average, not the organisational average.

This means that the centre of skill gravity for any team is unlikely to be on the w00t side of the scale.

In most large organisations the teams are more likely to be flattened pancakes across this bell curve taking in a reasonable representation of the organisation as a whole (especially as the highest skilled are often distributed amongst an organisation to attempt to bring up other teams.

This isn’t necessarily a problem though as there is value in diversity and individuals will each have different ways of thinking about things that can bring value to teams and organisations.

I’m beginning to think that the purpose of classic software process, and classic software process improvement is to try and move people along this scale from basic developer to software ninja, helping them to gain mastery through experience and feeding in the experience of others. When a team moves along this scale they begin to not need their process mentors any more and will seize their autonomy.

Technology is increasingly commoditised by innovation, as is software development and software process. I used to spend a lot of time teaching people Object Orientation but these days everyone just seems to know it, it’s nothing special or new, it’s just what people do. Similarly people are increasingly aware of the value of iteration, limiting work in progress, continuous integration and delivery. They need less process and less instruction as this stuff is becoming business as usual at least in more mature organisations.

As these problems are being solved I think that bigger problems are coming to the fore as they’re less hidden by low level development issues. The questions I see a lot of clients wrestling with are things like:

  • How can we foster the right kind of organisational culture?
  • How do we deal with requirements, architecture, releases, resources etc. in a complex system of systems environment?
  • How do we bring together multiple divergent interpretations of “agile”?
  • How do we manage outsourcing contracts in high speed agile projects?
  • How do we motivate and engage the business and technical communities?
  • etc. etc.

For me doing these things in a social collaborative way, that values individuals and teams, is Enlightened Software Engineering and encompasses the buzzphrase of the moment: Agile at Scale.

Solving these problems is less about technical skills and technical process content and more about social skills, psychology and understanding. Both from a coaching and business perspective.

So what’s my point?

My point is that software process improvement needs to focus less on individuals and more on teams, and teams of teams. That we should avoid ideology and take the best bits of knowledge and experience wherever we find them, growing our teams and individuals.

Also that we should apply more psychology to software process and business change finding socially resonant patterns for how we do things. True mastery involves not worrying about “breaking the rules” and from the outside can easily be confused with ineptitude.

Scaled Agility: The Project Forum

Name: Project Forum (Middle-out management structure) – Agile at Scale practice

When it might be appropriate

  • In situations where multiple competing stakeholder groups with different agendas are required to work together
  • In situations where multiple product groups need to collaborate on a bigger outcome
  • Where there is a conflict in direction, resource management/ownership or scope between collaborating groups
  • System of systems development

What is it?

The Project Forum is an application of agile philosophy to large project structures. Rather than impose a hierarchy of decision making from the Project Manager downwards the Project Forum is a virtual team in the middle of all stakeholders.

The Project Forum is a self-organising democratic group that balances competing voices and concerns, owns high level scope and architecture, runs the high level release train and performs integration activities for the product.

Use of the Project Forum practice does not prevent any communication directly between contributing groups it only provides a vehicle for that conversation when it’s relevant for the wider project.

From Traditional to Agile at ScaleThe Project Forum practice is an example of Agile at Scale combining social business practices, technical software practices and ways of working to make a simple way of doing big complicated bits of work.

Read more of this post

Follow

Get every new post delivered to your Inbox.

Join 344 other followers

%d bloggers like this: