Reduce waste: Visualise the value stream

A big thing in software process improvement (urgh) is reducing waste.  A great idea, but how do you identify waste?

One powerful method that I use is value stream visualisation. There’s a number of ways of getting information about the value stream ranging from physically walking the chain to simply asking people.

Sounds simple doesn’t it? But one of the best sources of information about wasteful processes and ways of working in a team is simply asking the team what they feel they shouldn’t be spending time on. By freeing up people to actually do their job they can reduce waste and improve productivity. One method I use is to ask each team member to identify a list of n items that they have to spend time on that they think are either entirely or partially wasteful. Basically a list of their impediments. I then anonymize the list look for commonality and then the top few are a reasonable target for improvement.

Alternatively actually walking the path of a piece of work through an organisation can really open the eyes to the sheer number of people and unnecessary waiting involved in some wasteful processes…

Anyway, having got some data I find it useful to visualise it in terms of waiting time and doing time. In Lean-speak cycle time is the time it takes to actually do something, lead time is the time it takes from the request being made to it finally being delivered. In many tasks the lead time far outstrips the cycle time, visualising it can make that very clear.

For example, a task with has a lead time of 5 days might actually be made up of 2 days waiting time before it actually gets picked up and the cycle time of 3 days starts.

If you look at all of the tasks in your value chain you might find that many of them look more like this, with very large wasteful waiting times. There are many reasons why there might be a large waiting time, typically the team doing the tasks is over-subscribed, or a team might be protecting it’s SLAs, KPIs or other TLAs. These problems can have significant effects when looking at the system as a whole.

Even more interesting is looking at how these tasks might fit together in a value stream. If we imagine a business function that involves a request moving through a bunch of tasks across a number of teams that have different waiting and doing times a simple sequence of tasks, with a little buffering between them from the Project Manager to allow for schedule variance naturally, might end up looking like this:

 

Here we have 26 days of actual work spread over almost 60 days (12 weeks) of elapsed time = 40% waste. Even if the person planning the work is willing to accept some risk and try to optimise their workflow a bit without improving the waste inherent in the tasks involved there’s still a lot of waste in the system.

By visualising the value stream in this way we can see straight away (apologies to red/green colour blind folks) that there’s a lot of red, a lot of waste. In many cases planners aren’t willing to accept the risks inherent in overlapping activities as shown here, or aren’t even aware that they can leading to the more sequential path shown above. The result is a minimum time that it takes a request to get done, based on the impedence of the current value chain of, in this case, 38 days before we even start thinking about actually doing any work.

Surely that’s not right.

What is “good” software architecture or design?

Architecture is a fine balance between a subtle science and exact art that combines cognitive problem solving, technical direction and expressing abstract views to aid common understanding. Design is similarly an inexact science which is inextricably linked to Architecture. As a result it’s quite hard to define what makes a “good” architecture or design.

There are high numbers of design and architecture practices ranging from the purely transformational to the evolutionary and emergent (both manual and automated) but there seems to be some heuristic consistency between these various practices and the approaches of the traditional, agile and post-agile movements.

Initially I saw the simple descriptions of that makes for “good” design from from Ron Jeffries and Kent Beck in Xtreme Programming. I like the style and direction of this kind of description so here’s my take on the characteristics that make for “good” design or architecture:

• Intentional structure and behaviour
• Highly modular: consisting of separate services, components, classes, objects or modules
  • Elements are highly cohesive
  • Elements are loosely coupled
  • Elements have low algorithmic complexity
• Avoids duplication
• Well described elements: modular elements have simple expressive meaningful name enabling readers to quickly understand their purpose, similar to code cleanliness we should strive for design cleanliness
• Runs and passes all defined tests or acceptance criteria
• Lightweight documentation

Intentional vs. Emergent Architecture

I’ve been thinking about architecture a lot recently but one thing that I often discuss but have never blogged about for some odd reason is intentional vs. emergent software architecture. Some old fashioned software methods such as waterfall led people into doing a lot of up front architecture work, they analysed and designed away for ages producing huge reams of UML and documentation that no one could ever squeeze into their heads if they had the patience to read it. This is an example of an intentional architecture – the architecture was intended, planned and deliberate.

Lots of folks hated that way of doing things as it meant people were writing docs and drawing diagrams instead of making working software, not to mention an all to frequent tendency to over-engineer architecture past the point of usefulness. This led to some people saying that we’re better off not trying to do any architecture and just letting it emerge from the work we do developing small little customer focused requirements (like user stories or similar).  Ok, so there’d be some rework along the way as we encounter a deeper understanding of the system and refactor our emergent architecture but it’d still be better than the old way of doing large upfront architecture.

So, there seem to be two opposed viewpoints: intentional architecture is best, emergent architecture is best.

For me, neither is true. I’ve seen some really terrible examples of badly over-engineered architectures that crippled a project and projects that never got past their over-reaching architectural analysis. Equally I’ve seen products with emergent architecture that had to be entirely re-architected as time went pay because their emergent architecture was so wrong it was comical (imagine a software management tool that only supports a single project, and that concept being deeply embedded in the architecture).

There’s a scale with intentional architecture on one side and emergent architecture on the other.

Various factors might push us one way or another… The second I listed on the right is interesting as if you’ve got a well known technology and problem domain you can get away with emergent architecture, but similarly if you have a totally unknown technology and problem domain it can be very effective to evolve towards a solution and architecture rather than crystal ball gaze by creating a (probably wrong) intentional architecture.

Which rather sums up the point I’m trying to make. The purpose of architecture is to shape the solution and address technical risks. Solving the big problems, creating common ways of doing something (lightweight architectural mechanisms) are all good architectural goals but only if we’re sure of the solution. If we’re not we’re better off evolving an emergent architecture, at least initially.

I think that the extremes at either end of the scale, as with most extremes, are a bad idea at best and impossible at worst. If you gather a group of people together and tell them to create a web app given a backlog but they’re not allowed to think or communicate about the architecture up front then you’ll find they all start dividing the problem in different ways and writing in different languages for different server and client frameworks. Hardly a good idea. Of course on the other end of the scale, believing that we can foresee all of the technical issues, all of the technology and requirements changes that might happen is as likely as a 12 month project plan Gantt chart being correct after a few levels of cumulative error margin have been combined.

For more on architecture see:

• Architectural profiling
• Architecture sketching
• Lightweight Mechanisms
• What is enough agile architecture?

Linux GUI Development: Lazarus 1.0 Review update

A while ago I wrote a review of Lazarus 0.9.30 and it came out with the 63% mainly because the installation process was so horrible. Well now it’s almost a year later and v1.0 has been released so it’s only fair I update my comments…

Installation

It’s still not exactly shiny but, and it’s a huge but, it is simple and it works! To install on Linux I had to download 3 .deb files (1 for Lazarus, 1 for the compiler, 1 for the source) and then I could install them by simply doing:

doing sudo dpkg -i *.deb

It’s worth tidying up any old Lazarus installs first which you can do by running my script here.

Downloading *.deb files and installing them is pretty normal for linux users so although it’s not a shiny wizard installer or single command it is simple and most importantly it just works! I’m going to upgrade this from 1/19 to 8/10

First Impressions

A lot of the little niggles have gone away and it’s a much more stable, solid feeling environment. It does still load up with a million windows (Delphi style) and feel a little old fashioned (a lack of wizzy GUI controls and UI customisation, single window mode etc.) but being open source and written in itself I can change these things if I want to reasonably easily. In fact loading up the AnchorDockingDesign package makes the IDE a docked single window affair and there’s plenty of 3rd part controls to download and play with.

Normal multi-window interface for Lazarus

Lazarus as a single window IDE

There’s still no multi-project support and recompiling the IDE to get a new component on the toolbar feels a bit weird even if it does work perfectly well. I’m going to stick with 6/10 for those reasons.

The GUI designer and Code Editor haven’t really changed since my last review and so they stay happily at 9/10.

Language features

The handling of generics has improved since I last looked at Lazarus and all seems to work pretty well now

Otherwise nothing much has changed, no  multicast events or garbage collection but those things don’t really slow me down much. That’s why I gave it 6/10.

Despite the lack of some of these things the language has always been and still is quite elegant (other than the nasty globabls), it’s got an excellent simple OO implementation and it’s really easy to quickly put an app together – oh yeah and it’s cross platform.

Cross-platform and backwards compatible

Lazarus works on windows, linux and mac. You can write a bit of code and natively compile it on each platform making for lightening fast code with no external dependencies. Write once, compile anywhere. You can already (partially) compile for android and other platforms are possible – anywhere the fpc compliler can be ported your code could work. That’s pretty impressive and it just works brilliantly.

Similarly all that ancient Delphi code can be loaded up edited and compiled and largely just works. Brilliant again. For that reason I’m going to bump up the language features to 8/10. (9 when android is working well).

I’ve not tried the feedback process again since v1 so I’m going to leave it at 7/10.

Conclusion

Despite it’s old fashioned feel in some places it’s simplicity is elegant and actually quite powerful. If you know the Object Pascal/Delphi language then it’s so fast to create good looking cross platform apps that it’ll knock your socks off.

When I pick up a new language I tend to do a challenge to load an xml file into a multi-column listview and create a details form to edit the selected row. In Java, something so apparently simple is a massive pain due to the imposition of the MVC pattern on everything. In C# it’s pretty easy, I can go MVC or not. In mono it’s reasonably easy too, although not the same easy and not as easy to get away from MVC.

In Lazarus it’s really easy, and it works fast and well on 3 major platforms. That’s a killer feature.

Category	Score
Installation	8/10
First Impressions	6/10
GUI Designer	9/10
Code Editor	9/10
Language Features	8/10
Feedback process	7/10
Cross-Platform	10/10
Overall	81% – Easy to use and powerful

In my previous review I said I couldn’t really recommend it, and that made me sad. Now I can recommend it, in fact it’s quickly becoming my technology of choice for linux gui development, because it’s quick to put things together and they look good. Just maybe Lazarus is living up to it’s vision and bringing Delphi back to life, but bigger and better than it was before!

Scaled Agility: The Project Forum

Name: Project Forum (Middle-out management structure) – Agile at Scale practice

When it might be appropriate

• In situations where multiple competing stakeholder groups with different agendas are required to work together
• In situations where multiple product groups need to collaborate on a bigger outcome
• Where there is a conflict in direction, resource management/ownership or scope between collaborating groups
• System of systems development

What is it?

The Project Forum is an application of agile philosophy to large project structures. Rather than impose a hierarchy of decision making from the Project Manager downwards the Project Forum is a virtual team in the middle of all stakeholders.

The Project Forum is a self-organising democratic group that balances competing voices and concerns, owns high level scope and architecture, runs the high level release train and performs integration activities for the product.

Use of the Project Forum practice does not prevent any communication directly between contributing groups it only provides a vehicle for that conversation when it’s relevant for the wider project.

The Project Forum practice is an example of Agile at Scale combining social business practices, technical software practices and ways of working to make a simple way of doing big complicated bits of work.

Simple software project measures

I’m not a big fan of metrics, measures, charts, reporting and data collection. I’m not terribly impressed by dashboards with 20 little graphs on showing loads of detailed information. When I’m involved in projects I want to know 3 simple things:

• How quick are we doing stuff?
• Are we on track or not?
• Is the stuff good enough quality?

There can be some deep science behind the answers to those questions but at the surface that’s all I want to see.

Organisations need to know that teams are delivering quality products at the right pace to fit the business need. To achieve this goal teams need to be able to demonstrate that their product is of sufficient quality and that they can commit to delivering the required scope within the business time scales. If the project goal may not be achieved then the business or the team need to change something (such as scope, resources or time scales). This feedback mechanism and the open transparent communication of this knowledge is key to the success of agile delivery.

The goal of delivering quality products at the right pace can be measured in many complex ways however, when designing the Project Forum agile at scale practice we looked at just 3 measures. In fact I should probably call them 2.5 measures as the throughput/release burnup can be considered mutually exclusive (if you’re continuous flow or iterative). The most important measure is people’s opinions when you go and talk to your team.

Note: in the measures section I often refer to “requirements” as a simple number, this could be a count, a normalised count, magnitude, points, etc. it doesn’t matter what’s used so long as it’s consistent.

The Kung Fu of Software Engineering

I’ve been studying both kung fu and software engineering for many years. I’ve come to realise that they are very similar and that kung fu is a pretty good metaphor for software engineering.

Done right it looks easy, but it’s not

When you watch kung fu in movies, or martial arts in general it makes sense logically, it looks sensible. Attackers punch in one direction and defenders block in another. Sometimes there’s tricks and special moves but they’re just a matter of learning them. However when you actually try and do these moves you find it’s not so simple. It’s not easy to react the right way under pressure when you’ve not done it before. You need to learn muscle memory, improve your fitness, work on your reactions and internalise sometimes counter-intuitive techniques. When you really do it well you use very little energy to do something that looks easy and it becomes easy but other people can’t get it just by watching you.

Both software engineering and kung fu are deceptively difficult, with hidden complexities and complex emergent behaviour. I think it was Grady Booch who said (although I couldn’t find the quote so any mistake is mine) “Software development has, is and always will be, inherently complex”.

Complexity built from simple small techniques

In many kung fu styles you learn basic small movements in repetition, often called “form” in martial arts, Sil Nim Tao (generally referred to as Siu Nim Tao in Wing Chun kung fu, the RUP of kung fu) is translated as “little idea form”. Learning this form we learn all of the basic movements and stances that set up the body positions required to get the mechanical advantage in a given situation. Each individual movement tends to be very, very simple.

This kind of information and learning is analogous to the basic software engineering knowledge that we give people. We teach them how to write in a language, idioms, patterns, standard architectures, frameworks, build technologies, iterative patterns etc.

However knowing which techniques apply in which situations, which play well with others and how to put them all together is another level of expertise based on experience. Teaching someone the basics does not make them a master. Software engineering, like kung fu is something you should never stop practising and learning.

It gets more complex when you add more people

Defending yourself from one attacker is a whole different ball game than defending yourself from two attackers. Defending yourself from a group of attackers breaks down the whole mixed metaphor of ballgames, sports and anything else in the vicinity. The complexity of the action increases significantly as you add more people, it’s not just a linear relationship. As more and more people are involved there are emergent behaviours that can’t be predicted from the beginning.

This is true of any activity that multiple people take part in, especially complex activities. In kung fu it means you have multiple attacks, more energy in your attackers which means once you’re tired you’re in trouble. In software it means you have multiple people doing things at the same time with subtly or radically different ideas on what should be done and the best way to do it.

The only ways to reduce this complexity in software engineering are to go up or down. We can either abstract away from the complexity moving to higher level technologies where possible (sacrificing fine control typically) although such abstraction tends to bring it’s own complexities or we can dive down and educate the team (in the broadest sends) on the complexities to try and reach a common understanding.

No plan survives contact with the enemy

Trying to plan in detail all of the details of a kung fu fight, even against a known assailant is about as pointless as trying to plan all of the details of a software project. There is too much uncertainty, too much complexity and too much emergent behaviour. Above all there is too much change. In both kung fu and software engineering we need to remain agile and responsive to change, in the environment, the different things being thrown at us and our own actions.

There’s no magical solution

We’re not in the Matrix, we can’t download kung fu skills into our heads in seconds. Or software engineering skills. These things take years to learn, will be slightly different for every individual as they tailor the standard wisdom to their particular individual skills and style.

There’s a lot to learn. Personally I work as a software development coach helping people structure and plan their work from architecture and design mentoring, SCM & Build techniques and tooling, requirements management, agile and iterative project management, portfolio and business management. In many ways these kind of things, and others similar to them, can be thought of as different styles of martial art. Just because you’re good at one of them doesn’t mean that you’re good at another, or the next new one that comes along. Of course a certain aptitude helps, and knowledge of one certainly makes others easier to learn but beware of “experts in everything”.

We can’t all be Bruce Lee, Jackie Chan or Jet Li but we get a choice about where we are on the spectrum between being a master and an armchair expert sitting on the sofa watching others do it.

Finally

I believe that there’s an exact art and subtle science to both martial arts and software engineering. We need to practice these skills, we need to be continually learning and improving. We need to learn from other styles and experienced practitioners.

“Kung Fu” is actually translated to “achievement through great effort”

If you’re in the Cheltenham, UK area come and do some kung fu with me at Chi Wai Black Belt Academy.

I’ll leave you to make your own Chuck Norris software jokes…

First Look: CLM 2011 RTC + RRC + RQM 3.0.1 review and screenshots

What is it?

CLM 2011 is a suite of tools making up IBM Rational Team Concert, Rational Requirements Composer and Rational Quality Manager properly integrated for what is arguably the first time. In my opinion this release is the closest to my interpretation of the original Jazz vision that IBM have delivered yet. It’s not perfect but I like the direction it’s heading in.

It’s taken me a while to write this, mainly because I’ve been very busy but also because it seemed to take some deep magic to get all of this stuff installed and running properly. But I’ve done it a couple of times now so I thought I’d post some screenshots and comments. Of course if you look back on my posts of the original RTC review you’ll see that life is harder these days already. With RTC 1 and 2 you could just download a zip, extract, run a script and you were off and away. Installation isn’t too bad… it’s not great and simple but this is a big complex suite of tools so I don’t really expect it to be trivial.

Lifecycle Project Administration

To start with, the management of projects across these separate tools has been significantly improved by the introduction of Lifecycle Project Administration which allows you to create an integrated cross tool project in one step and simply manage members and roles across them. This is a big step forward although there are still some problems in that it’s not easy to do these kind of things at scale. For example if I want to add 3 people with a set of roles to 40 projects I’m going to be doing a lot of manual clicking. In fact generally it’s not so easy to deal with project areas at scale in the Jazz toolset and that hasn’t significantly improved yet although Lifecycle Project Administraton is an important step in that direction.

Creating a new Lifecycle Project

A Lifecycle Project

Editing roles accross tool project areas

I’m not a big fan of the navigation between project areas (linked or unlinked) as the user needs to understand the architectural relationship between personal dashboards, Change and Configuration Management (which has the RTC bits like plans, work items, source control, builds etc.), Quality Management (which has the RQM bits like test plans, cases, scripts etc.) and Requirements Management (which has the RRC bits like diagrams, UI sketches, docs, requirements etc.) to navigate the stuff in their single project. I think it’s a mistake to build the UI navigation around the architecture, I would prefer to see a unified project interface and navigation structure with the extra products adding to the project mass like aspects on a foundation. As before this becomes even more of an issue when you scale up to hundreds of projects. Incidentally, aspect orientation is how we apply practices to process kernels while still providing a seamless user experience of composed practices.

Cross (linked) project navigation

So although the three products are closer together than ever before, sharing a UI and a common architecture they still are separate products and that’s clear in terms of navigation, UI differences  and linking items between them. This is a shame for many reasons but one of the most important is that it’s still providing niche tools for separate roles, building walls between people in development teams and making cross-functional teams harder to create as individuals have to learn specific skills. These differences are not a strength, they make the whole game of software development harder.

To be fair though I’m yearning for an ideal perfect solution, CLM 2011 isn’t that idealised perfect solution but it’s a lot closer to it than anything else I’ve used!

Let’s start with IBM Rational Requirements Composer

RRC 3.0.1 is a totally different beast than RRC 1 (see my original 2008 review here) and shouldn’t be thought of in the same light as RRC1. The old version was eclipse only, badly integrated, dependent on IE and full of bugs – this version is entirely web based, deeply integrated into the Jazz platform and not dependent on IE! As for bugs, I don’t know yet but it’s actually usable which v1 wasn’t!

What it does:

• Tracks requirements of different types with different attributes and links
• Web based diagram editing (via browser plugin for FireFox or MSIE ), although someone used Microsoft CAB technology inside the FireFox xpi so don’t expect to do any diagram editing from linux or a mac
• Ability to produce use case diagrams, UI prototypes, UI storyboards

For me this is the first time RRC has lived up to some of my early hopes for it. I see this as a replacement for ReqPro, and indeed a replacement for DOORS in time.

Creating Requirements Artifacts

Linking a requirement to a new implementation story

Unfortunately I only use linux at home so I couldn’t take screenshots of the RRC web editor, even in a virtual machine running windows on an explicitly supported browser I can’t get the plugin to work. When I’ve used it my office environment I’ve really quite liked it though, although it’s got quite a lot of basic usability problems. I’m looking forward to when it’s more mature.

There is also a dependency/traceability tree viewer but that’s a separate download at the moment.

Next Implement something in IBM Rational Team Concert

RTC hasn’t changed that much from a user perspective, it’s still a great tool for tracking work items, making atomic multi-file change sets in parallel with continuous integration and build management. I’ve been using RTC in anger for 3 years now and still like it and think it can help drive agile benefits in development teams. Granted it’s got some issues like any tool, enterprise scalability issues and annoyances the biggest of which is the lack of cross-project queries which is practically unforgivable from my perspective. See this work item on jazz.net and demand cross-project queries from IBM!

With that said, working in Eclipse from workitems, fully using Jazz SCM and continuous build is awesome.

Edit a work item in RTC	Sprint Plan in RTC	Making code changes in RTC Eclipse
Looking at build results in Eclipse	Looking at Build results in the web

Then Test via IBM Rational Quality Manager

I’ll admit it, I’m not a cross-functional ideal. I’m a typical developer, I’m not a tester, nor am I a test manager so I’ll leave the finer points of testing theory to others. However I do have some things to say about RQM from a cohesive logical process perspective.

In this example I’ve created a Test Case based on the pizza requirement I entered in RRC (in exactly the same way that I created the implementation story). At this point frankly I’m a little disappointed because my Test Case has been created and is linked to the requirement (good) but it has no knowledge of the implementation story that I also created from RRC.

Test Case in RQM

Test Case link back to requirement

No pre-existing link to development items

The Missing Link

For me this is the missing link. I realise it’s not as a simple as their always being a related triangle of 1-1 associations between these different types of artifacts, but the link between the test and the development items should at least be suggested as the odds are fairly strong that if a requirement is developed by an bit of code the test for the requirement is likely to need to test the aforementioned bit of code. Obviously I can manually create this link but that’s not the point.

In fact this is symptomatic of the fact that these CLM tools are still separate and then integrated. There is a separation between requirements in RRC, development plan items and tasks in RTC and test plan items and tests in RQM. I have to create work items /artifacts  in each tool to represent these different things and link them all together. Which is not really what I want to do.

I don’t want to spend a lot of my time creating at least 2 items for every requirement in my product (1 dev story+tasks and 1 test case+scripts).

What I want to do is elaborate a set of requirements in a simplistic user friendly UI with nice diagrams (RRC can do this) then view that candidate backlog immediately in my planning tool and further refine it – not copy it (even in bulk) to RTC but to actually manipulate the same work items, albeit a different aspect of them, in RTC. I want testing to be an inherent part of development with quality management and assurance being just another aspect of the system. Every requirement should have quality dimensions and be a test case although I might want to add more of course.

I want to have requirements with dev tasks and tests hanging off them.

Basically I want to define something that needs to be done, plan it, do it and test it all as one high level thing. I might drop off loads of tasks, different types of tests, supporting sketches etc. but I want a holistic understanding of a development item with the ability to project different views to different types of stakeholders (customer requirements views, test professional views, management views, development views).

Some conclusions

CLM2011 is a serious step in the right direction from IBM. In my opinion Jazz always has been but it’s been suffering from too many silly little problems and a lack of meaningful deep integration (ironic considering the original mission). If I had a magic wand I would do the following feats of greatness to the Jazz platform which I believe are all necessary to turn what is a good suite of tools into a true killer software development environment.

• all jazz tools to apply new aspects to projects rather than creating seperate project areas which then need linking and administration via LPA
• all artifacts and workitems to be viewable as native items in any tool interface
• all artifacts and workitems created by all three tools (and other future ones) be instantiated and linked with truly consistent UI, all taggable and commentable
• all artifacts, scm/build elements and work items queryable via a single query engine (across linked and un-linked project areas)
• the ability to self serve projects (without granting massive impractical rights), finer grained security and permissions control
• parent-child change sets
• smoother flow of work items between tool aspects rather than creating new ones with links between them
• make RRC diagram editing work on linux – things that only work on Windows are not enterprise deployable, and if they’re browser based and only work on windows someone needs shouting at. Even a much maligned school boy shouldn’t be making this error
• a decent reporting solution and technology

Finally , being able to capture and elaborate requirements, plan iteratively, develop code, continuously integrate with unit tests, system test and manage test plans (amongst other things!) in what feels close to a single tool is extremely powerful. If only navigation wasn’t architecturally focused this strength would be much stronger and be a killer feature for IBM and Jazz.

If they sort out cross-project querying.

How to avoid Fragile Agile, Flexibility in Context

A presentation I gave at the UK RUG Annual Meeting on what can make agile development fragile and how to avoid that fragility.

Unfortunately animations don’t work and slideshare’s screwed up the agenda slides but you can still follow it

Why do people still waterfall their development process?

Generally the software development industry has caught on to the fact that iterative development is a good thing. Only people that don’t believe in common sense think it’s a great idea to leave all of your testing to the end and hope for the best. With that in mind then why do people still have a waterfall mentality to their software development processes?

At the beginning of a project you can’t possibly know everything about it, that’s one of the many reasons it makes sense to iterate, so you can do some work and then refine your plans based on your work. For some odd reason though a lot of projects, even though they might develop iteratively, seem to think they should define their process at the beginning of a project in great detail and then stick to it regardless of the facts due to some misguided need for process alignment or similar.

That attitude just doesn’t make sense. A view of what practices will be followed is necessary at the beginning so that everyone’s on the same page and now what they’re meant to be doing, but even the formally defined iterative processes like RUP say that the process should be examined and refined each iteration.

In my experience an extremely detailed formal process is difficult to change. If you’ve got a 50 page development case. huge stack of xml files in your favourite process authoring tool or similar it’s very hard to make useful changes to a process. Of course you can change the review requirements of the Use Case Specifications from Formal-Internal to Formal-External, but it’s not so easy to scale down the formality of business modelling while adding a practice to address user experience issues.

Software development process can (and should) be thought of as a collection of practices. Each practice should address a particular aspect of software development in a complete fashion. Every small technique is not a practice (in coding terms every idiom is not a pattern). Using Use Cases, being Iterative,  using Component Based Development are all good examples of practices. They can be added or removed to your project’s way of working and the level of formality can be scaled up or down on each practice. Practices are the natural unit of adoption for processes.

It’s always best to start small and simple. At the beginning of a project just do what you can see. It might initially make sense to think about using Use Cases as a common way of managing requirements for your project and it clearly makes sense to iterate in most cases. Then as the project progresses and more understanding is reached more practices can be added as necessary. This is an iterative approach to software development process, it’s just common sense.

At IJI we’ve been doing this sort of practice based way or working for ages – in fact it’s what the company was created around as the fine brains we have created this approach and brought it to life in EssWork and EssUP which is a kernel and a collection of practices.

Anyway, enough of the company blurb, more about practices. I think everyone can agree that this is a sensible approach to software development as it applies the lessons the industry has learnt about software development back onto the problem. It doesn’t matter which vendors tools are used, or what process frameworks are used this is just a sensible approach, and the best thing is it is compatible with whatever you’re already doing that is successful or necessary in your organisation. I work with organisations helping them to change and adopt their processes and in every organisation I’ve been to is doing at least one thing well. Using practices to work with process improvement means that we can just add to what’s already working, incorporate the necessary governance or operational activities in an organisation, add SCRUM if that’s what’s wanted and have it all work together in a cohesive way of working.

To achieve this adding together of practices in a way that works you need to have a foundation that provides a common understanding of the basic concepts of software development so that practices can fit together neatly. This is intellectually quite tricky to achieve but necessary. It should also be practically invisible to the people doing projects – they don’t want to think about process geeky things like the process architecture that can support practice composition in a seamless way. At IJI we call this the kernel – which I think is a suitably geeky term. We have also built it both conceptually and in software (that’s what EssWork is) along with what we think are the “essential practices” (that’s what EssUP is) and now the big vendors are following suit and packaging their processes this way.

Maybe one day we can get away from branded processes by vendors and just have a software development community using, sharing and collaborating on practices.

Oh yeah, and they should be in tools not websites, and free!