Mike MacDonagh's Blog

Somewhere in the overlap between software development, process improvement and psychology

Tag Archives: RUP

Launch: Holistic Software Engineering

How do we join up business strategy to agile development? Is program management relevant? Where do project managers fit in? What about architecture?

Holistic Software Engineering (HSE) answers all of these questions – for free.

Agile and continuous flow are great for small teams or a small number of inter-related small teams working on exploratory or maintenance work. But what if we’re spending 100s of millions on an IT strategy of inter-related products that need to work together to deliver business value. What is business value anyway?

H-Model To answer these questions (and more) my friend Steve Handy and I have distilled our collective 30+ years of software experience in a single, cohesive model of software development. We’ve developed the H model that moves on from the v-model and it’s siblings by combining:

…all elegantly combined and de-conflicted by release planning.

We’ve not invented very much, we’ve simply put a lot of good ideas from others together into a cohesive framework. We’ve drawn it all as a big picture and developed a website that shows how to get value from all of this stuff. Everything is clickable, everything has content.

The best bit: it’s free! There’s no paywall, there’s no private “full” version, you can just use it or not as you like.

We don’t believe in process zealotry, or putting academic concerns above clarity and usefulness. HSE is indicative, not prescriptive. You’re already doing it and if you use the big picture to draw red blobs on the bits that aren’t working well, or missing, in your organisation then you can use the model to make tangible improvements – immediately.

Using HSE doesn’t replace any of your existing processes, it provides the glue that joins them all up together in one simple, elegant and cohesive model.

Holistic Software Engineering
is for large/complex software organizations
who need to understand modern software engineering in a business context
our solution is to present a big picture of software business backed up with practical detail avoiding academic or heavyweight process documentation
that covers people issues, business strategy, portfolio, programme and project management as well as architecture, technical delivery and integration
unlike simple small team based processes such as RUP, Scrum etc.
The big picture of software engineering

Holistic Software Engineering

And if it’s too big, we’ve got a small picture, which is essentially “normal” agile development.

Please share HSE with your friends and colleagues.

I’m coming out as not-Agile and not post-Agile

Big A vs. Little a

Huh? What? I’ve written a fair bit on this blog about agile topics, but I always try to write about agility with a small “a”. I’m not really into Agile with a big “A” though – I’m not into doing things according to a set of rules and having arguments about whether I’m doing it right or not. I’m not anti-agile, but I’m increasingly anti-Agile.

To me, the ideological arguments, definitions of everything, frameworks, money-spinning certifications and money-spinning tooling are what’s wrong with doing “Agile”. Being empirical, reflecting and adapting, honestly communicating and putting people first as an approach is being “agile”.

I don’t really like the term “post-Agile” either though as it comes with a bunch of baggage and is easily misinterpreted – and I still see benefits in adopting agile practices. I don’t want to see another specific set of rules or a statement or beliefs with elitist signatures. For me what’s next after agile is about dropping the ideology in software process, destroying the ivory tower of trademarks, formal definitions, money spinning tools and money-spinning certification programmes.

We need to get rid of the League of Agile Methodology Experts and if anyone says “Ah, but this is different” when showing a website of process content then you have my permission to hit them with a stick.

So what does the future look like?

Software development is a complex social activity involving teams of people forming and self-organising to work together, sometimes in teams of teams which is even harder. As technology is increasingly abstracting up and raising in maturity so is the way that developers, managers and organisations think about software and doing software. I think the problem is getting increasingly social, and the solutions will start looking increasingly social using more “soft practices”.

Software process improvement agents/consultants/coaches/mentors (including myself) need to take a long hard look at themselves. Are they telling people how to do it right when they can’t even write HelloWorld in a modern language? I’ve said that to some acquaintances in the industry, generously qualifying “modern language” as something significantly commercially used in the last 10 years and seen them look quite offended at my insulting affront to their professional integrity. I’ll go out on a limb and say you can’t coach a software development team if you don’t know how to write software.

So… software process improvement?

TOverlapping concerns in process improvementhe world runs on software, it’s everywhere and it’s critical. Getting better at doing software, improving the software value chain, is a noble aim and will continue to be as it means getting better at doing our businesses and our lives.

For me process improvement (dislike that phrase as well) is going to be more about combining psychology based business change practices with the best bits of a wide variety of ways of working (agile, lean, Scrum, what you currently do, RUP, Kanban, various workflow patters etc.) with technical software development practices like continuous integration, continuous delivery.

We need to work together, not as “leaders and followers” or “consultants and clients” but as collaborative peers working together to apply specialist skills and experience so that we can all improve. Smart people should be treated as smart people, we have much to learn from them and should be thinking in terms of fellowship rather than leadership.

I’m calling this overlap “soft practices”* because the term is evocative of:

  • The practice of doing software
  • The practices involved in doing software
  • Being able to deal with people is sometimes called  having “soft skills
  • Soft power

What do you think about “post-Agile”?

* I’ve even set up a company called Soft Practice to do this stuff, that’s why I’ve not been blogging much recently, who knew there’d be so many forms to fill in!

Edit 14/8/13: Seems others are now talking about the same things: Ken Schwaber, Dave Anderson, Aterny

Get your point across using positive statements

This is a simple one but amazingly important: use positive statements.

A positive statement is something like “to sprint you must run fast”. Conversely a negative statement is something like “to sprint you must not run slowly”. The problem with using negative statements is that for them to be processed by the listener the concept being negated (in this case running slowly) has to be represented first by the listener before being negated. Here’s two examples, one about software and the other about addiction. Then I’ll cover the more devious application of the technique before finishing with the positive application.

If you tell someone not to do something first they have to think about doing it, a bit like my purple penguin example in the introduction to this blog series.

Software

Here’s an example from software process training I used to hear regarding RUP phases: “The RUP phases Inception, Elaboration, Construction and Transition are not requirements, design, implementation and deployment…” This was a well intentioned phrase aiming to dispel a common misconception when risk-value lifecycles were first being introduced to classic waterfall minded managers. Unfortunately this sentence actually reinforces the misconception one and maybe two reasons, firstly it’s a negative statement meaning the listener has to make that link in their mind between these phases and the waterfall stages and then negate it. The second reason is that it’s too easy for a tired person on a training course to miss that key “not” word and totally misunderstand the trainer’s intention.

This happened to a colleague of mine who used this sentence and the client’s training manager sitting on the course mis-heard (swore they didn’t when challenged) and never used my colleague’s training services again.

Addiction

Imagine someone who’s going to give up smoking. Friends with decent intentions may say “Just don’t think about smoking” and yet the end result is that the listener has to think about smoking (linking in all of their anchored states).  In fact there’s a whole industry of negative statements around giving up smoking that are designed to make people think it isn’t an easy thing to do.

My first sentence might sound a little odd. Normally the phrase is “trying to give up smoking” but that’s because the word “try” presupposes failure which is why marketing folks have done their best to ensure it’s the normal way of talking about giving up. Just think of all of those adverts for giving up aids that tell smokers how hard it is. Sneaky.

The Sneaky Side

The problem is that a negative statement sneaks in a sliver of representation that counters the stated meaning. There’s a really insidious side to this when considering negative questions. Negative questions are any question including the word “not” (even in the form of a contraction when it’s even more sneaky) and they are a clever way of embedding pre-suppositions. Here’s an example:

“Don’t you think using positive statements is a good idea?”

Expand it out and it becomes:

“Do not you think using positive statements is a good idea?”

The second form of the question sounds very odd and unnatural. In fact it’ll confuse many people for a moment (excellent pattern interrupt) and yet the first form doesn’t sound so strange, and yet rather oddly people will almost always agree to a statement using a negative contraction. Magicians and mentalists use this trick a lot “Couldn’t you have changed your mind at any point?”, “Haven’t I been reasonably honest about …?”. And of course if the entertainers are doing it you can be sure the sneaky salespeople, callous cold-readers and other such charlatans are doing the same. In fact this particular construct is used especially heavily by cold readers, the effect is multiplied based on the level of rapport at one far end of the see-saw even slight nudge is enough to direct a listener.

If you consider the basic form of the question “Do you think using positive statements is a good idea?” you’ll notice that’s it’s neutral, it doesn’t presuppose an answer in any direction. That’s the difference.

The Positive Side

I’ve written a lot on the ethics of using somewhat sneaky linguistics before so I’ll leave out that discussion again. But there are some great positive ways to use this knowledge. Here’s an example from this very post (+10 internet points if you noticed it earlier)

In the section about addiction I wrote: ” there’s a whole industry of negative statements around giving up smoking that are designed to make people think it isn’t an easy thing to do”. I deliberately used a contracted negative statement that presupposes that it is an easy thing to do. I used a negative statement to cause a positive representation.

Another simple way to use this knowledge is to simply use positive statements. Tell people what you want them to do and think. Avoid telling people what not to think. (er.. two negatives in one sentence what the…?)

This works especially well with children, instead of telling them not do something (such as “stop being noisy”) tell them what to do positively (such as “be quiet”). The first makes the kid think about being noisy, the second makes them think about being quiet. However it must be said that even the most carefully crafted sentence isn’t going to get a little kid to be quiet, but that’s ok they’re meant to be noisy.

Finally

So, aren’t you glad you read this?


This blog is part of a series on Holistic Communication: The linguistics of business change. Introduction, ethics and table of contents is all in the first post.

Definition of done

Teams are sometimes split into smaller sub-teams. Almost always they are part of a bigger team, which in turn is part of a bigger team and so on and so forth until we reach the organisational entity. Don’t get me started on B2B teams.  Imagine an onion, with lots of layers. A weird mutated onion with multiple cores and overlapping onions. One that has tendrils sticking out into other similarly mutated onions like a big brain… Ok metaphor taken too far. Back to the topic…

The point is that everything team is connected to everything else. At every boundary it’s necessary to have a “Definition of Done” so that teams and their stakeholders understand what they’re getting.  Any good or otherwise set of software development practices (like Scrum, RUP, waterfall  or whatever you’re using) should help the team define the levels of their Definition of Done. I’m not using a specific example so I’ll just make some up off the top of my head to illustrate the point:

  • Fully documented, discs printed, shrink wrapped and online distribution site fully functional, support systems in place ready for users.
  • Beta test program
  • Acceptance Tested
  • Integration Tested
  • System Tested
  • Unit Tested
  • Code Built
  • Code written
  • Code designed
  • Idea thought of
  • Requirement captured

Given some levels like this we need to ask the question where do we draw the line? As an R&D team, as a component team, as an integration team or business team I’d expect different answers. And for those answers to vary according to points in the lifecycle and relationships between teams and stakeholders.

The point is that these different levels of done take different amounts of work and are intended to differentiate different levels of quality (although I’ve used a fair few public release versions of software that don’t appear to have even been unit tested!). Without working out what the definition of done is between teams or parts of a team no one can estimate, plan or deliver any work.

If you have problems with multiple teams working together in your organisation, a lack of understanding of the Definition of Done is one of the common causes and is relatively easy to fix.

Use Case vs. User Story

Use Cases are too big to fit into a sprint/iteration! User Stories are so fine grained there’s too many too keep track of! Where’s the big picture? How to we define releases? Argh!!! I don’t know which to use!

Personally I tend to use both. I don’t think there’s any conflict between Use Cases and User Stories, in fact they’re rather complementary. Here’s how and why….

A few years ago I delivered a presentation at an agile software development conference entitled “Do requirements still matter?”. The short answer was “yes”. Most descriptions of agile iteration start with having a backlog, prioritising etc. etc. But where does the backlog come from? My answer: the Backlog Fairy; obviously.

The Backlog Fairy

Of course requirements still matter, they’re how we get a common understanding between customers and development teams of what we need to do. They’re the units by which we can incrementally build systems. In the rush to adopt agile processes a lot of teams have forgotten that they still need to start up with some lightweight analysis of scope and risk.

One of the things Use Cases are great at is capturing the big picture, showing a simple diagram of stick people and blobs that people who’ve never heard of UML can happily discuss in terms of what big bits of functionality are in and out of the system and who’s going to do them. The humble Use Case diagram. Not so easy to do that with a massive list of small stories.

However Use Cases can often be a bit too big to fit into a single development cycle but since they’re made up of a lot of different scenarios they’re easy to slice up into smaller bits. This is where I often use stories.

The advantage of Use Cases defining the scope is they’re nice chunky things to estimate and prioritise in a first pass. Of course as we get going we’ll estimate and prioritise stories for sprints/iterations but Use Cases help us prioritise requirements into Releases.

So I use a Use Case diagram to describe the high level scope of the thing to be developed. This normally takes about 5minutes to sketch. Then I use the Use Cases identified by the diagram (not documents, just the ovals on the diagram) to focus discussion on deriving stories (or use case slices) for development and testing within a sprint/iteration.

Sometimes stories turn up that don’t really fit into the early Use Case model, this is a rather good thing as it lets us challenge the understanding of scope. Does the story not fit in because it’s not really in line with the customers priorities and needs, it’s a cross-cutting or architectural concern or because we’ve missed some important part of the scope? All are important things to understand.

The Kung Fu of Software Engineering

I’ve been studying both kung fu and software engineering for many years. I’ve come to realise that they are very similar and that kung fu is a pretty good metaphor for software engineering.

Done right it looks easy, but it’s not

When you watch kung fu in movies, or martial arts in general it makes sense logically, it looks sensible. Attackers punch in one direction and defenders block in another. Sometimes there are tricks and special moves but an observer can see the logic in them, and they are feasible if clever – they are complicated, not complex. However when you actually try and do these moves you find it’s not so simple. It’s not easy to react the right way under pressure when you’ve not done it before. You need to learn muscle memory, improve your fitness, work on your reactions and internalise sometimes counter-intuitive techniques. When you really do it well you use very little energy to do something that looks easy and it becomes easy but other people can’t get it just by watching you.

Both software engineering and kung fu are deceptively difficult, with hidden complexities and complex emergent behaviour. I think it was Grady Booch who said (although I couldn’t find the quote so any mistake is mine) “Software development has, is and always will be, inherently complex”.

Complexity built from simple small techniques

In many kung fu styles you learn basic small movements in repetition, often called “form” in martial arts, Sil Nim Tao (generally referred to as Siu Nim Tao in Wing Chun kung fu, the RUP of kung fu) is translated as “little idea form”. Learning this form we learn all of the basic movements and stances that set up the body positions required to get the mechanical advantage in a given situation. Each individual movement tends to be very, very simple.

This kind of information and learning is analogous to the basic software engineering knowledge that we give people. We teach them how to write in a language, idioms, patterns, standard architectures, frameworks, build technologies, iterative patterns etc.

However knowing which techniques apply in which situations, which play well with others and how to put them all together is another level of expertise based on experience. Teaching someone the basics does not make them a master. Software engineering, like kung fu is something you should never stop practising and learning.

It gets more complex when you add more people

Defending yourself from one attacker is a whole different ball game than defending yourself from two attackers. Defending yourself from a group of attackers breaks down the whole mixed metaphor of ballgames, sports and anything else in the vicinity. The complexity of the action increases significantly as you add more people, it’s not just a linear relationship. As more and more people are involved there are emergent behaviours that can’t be predicted from the beginning.

This is true of any activity that multiple people take part in, especially complex activities. In kung fu it means you have multiple attacks, more energy in your attackers which means once you’re tired you’re in trouble. In software it means you have multiple people doing things at the same time with subtly or radically different ideas on what should be done and the best way to do it.

The only ways to reduce this complexity in software engineering are to go up or down. We can either abstract away from the complexity moving to higher level technologies where possible (sacrificing fine control typically) although such abstraction tends to bring it’s own complexities or we can dive down and educate the team (in the broadest sense) on the complexities to try and reach a common understanding.

No plan survives contact with the enemy

Trying to plan in detail all of the possible interactions of a kung fu fight, even against a known assailant is about as pointless as trying to plan all of the details of a software project. There is too much uncertainty, too much complexity and too much emergent behaviour. Above all there is too much change. In both kung fu and software engineering we need to remain agile and responsive to change. Change in the environment, the different things being thrown at us and our own actions.

There’s no magical solution

We’re not in the Matrix, we can’t download kung fu skills into our heads in seconds. Or software engineering skills. These things take years to learn, will be slightly different for every individual as they tailor the standard wisdom to their particular individual skills and style.

There’s a lot to learn. Personally I sometimes work as a software development coach helping people structure and plan their work from a business, planning and architecture perspective mixing in  SCM & build techniques, tooling, requirements management, agile and iterative project management, portfolio and business management. In many ways these kind of things, and others similar to them, can be thought of as different styles of martial art. Just because you’re good at one of them doesn’t mean that you’re good at another, or the next new one that comes along. Of course a certain aptitude helps, and knowledge of one certainly makes others easier to learn but “experts in everything” are few are far between.

We can’t all be Bruce Lee, Jackie Chan or Jet Li but we get a choice about where we are on the spectrum between being a master and an armchair expert sitting on the sofa watching others do it.

Finally

I believe that there’s an exact art and subtle science to both martial arts and software engineering. We need to practice these skills, we need to be continually learning and improving. We need to learn from other styles and experienced practitioners.

“Kung Fu” is actually translated to “achievement through great effort”

If you’re in the Cheltenham, UK area come and do some kung fu with me at Chi Wai Black Belt Academy.

I’ll leave you to make your own Chuck Norris software jokes…

Ngrams for nerds

Pictures that are worth 500 billion words!

Google Ngram Viewer shows graphs of how many times words or phrases have occurred in a set of 5 million books over the years. They’re a really interesting way of seeing trends in information and relative importance between words. It’s free and easy so check it out.

Here’s some I recently ran that I found interesting. I ran most of them from 1950 onwards and  the info only goes up to 2008.

Comparison of programming languages

Programming Languages

Ngram link – When looking at this you’ve got to mentally remove the baseline Java and Pascal references from the 1950 as they’re about coffee, islands and mathematicians. Interesting to see Java so dominant.

Programming paradigms

Programming Paradigms

Ngram link – I found this one really interesting. Compared to the others in my query “structured programming” had a lot more books written about it. I wonder how much this is a reflection of the rise of the internet… these days although there are lots of programming books the primary source for learning a language is online material?

Methodologies

Methodologies

Ngram link – I was a little surprised to see RUP so much more prevalent than agile but then I did have to add “software development” to the term to avoid including the bendy and stretchy. Also as with the previous one I suspect that there’s a difference here between a vendor driven process with supporting books and a more open source philosophy on agile as a generic umbrella for methodologies, and therefore more online sources. As Ivar Jacobson says: “No one reads process books

Shareware, Freeware and OSS

Shareware, Freeware and Open Source

Ngram link – This one speaks for itself 🙂 I wish I could have worked out how to add “expensive vendor products” to the query!

User Stories vs. Use Cases

User Story vs. Use Case

Ngram link – Ah yes, this argument again. Interestingly this dominance of use case over user story in written books correlates with query stats between user stories and use cases on by blog and the ivarjacobson.com site. Personally I think they’re both great and complimentary, I often use them together on software projects.

Windows vs. Linux

Windows vs. Linux

Ngram link – Yep, Linux beats Windows at every turn.

More Ngrams!

For more fun with Ngrams watch this very funny video explaining this stuff

How to avoid Fragile Agile, Flexibility in Context

RSDC 2008

The RSDC (IBM Rational Software Development Conference) is on in the first week of June in Orlando, Florida. I’ll be there with a number of IJI colleagues including of course Ivar Jacobson. This year I’ve got two talks:

The big questions on my mind are, what’s William Shatner going to talk about (should be fun) and where’s the special event this year? Does anyone know?

This year I intend to not get humiliated on the mini-golf course while sun burning my ears.

RSDC banner

%d bloggers like this: