Mike MacDonagh's Blog

Somewhere in the overlap between software development, process improvement and psychology

Tag Archives: continuous integration

Testing is dead

Long live testing!

There’s an apparent conflict between the idea of a cross-functional team collaborating together on their work and specialist testers that only do testing. There’s an awful lot wrong with the “them and us” attitude between developers and testers (in both directions) in some organisations. One of the biggest problems in isolating testing from development is it abdicates responsibility for quality from the developers, it makes a poacher-and-gamekeeper culture driving divisions between people. There’s a well established cliché of development chucking rubbish over the wall, there’s also a well established cliché of testers being empire building bureaucrats who only ever say no.

The biggest problem from my point of view in having separate developers and testers is that it creates two teams (an example of Conway’s Law), a team of teams  and that causes all sorts of problems with communication, identity, culture, decision making, definition of done etc. If you’ve got a small team and you split it into two by having devs and testers you’re taking something that is potentially high performing and deliberately interfering to make it less likely to succeed. A typical management activity.

Testing as part of the team

In my teams we’ve always considered that we have to build quality into every activity if we have a hope of achieving quality in the product. That means our lowest levels of done aren’t that we wrote some code, or did a build but that we have done some testing. We have tested the basic and alternate flows, or the obvious paths through the stories. We don’t consider it stable/promoted/out of development until this has happened.

Ideally I like to do this two ways… using automated tests as part of my continuous integration to test the obvious paths and the various parameters that impact the system (checking different data sets against expected functionality etc.). The other is for the team to test each others stuff – not with a test script but with the requirement/acceptance criteria. This allows us to check what isn’t “normal”, some of the fringe cases, but mostly to check that our view of “normal” is consistent across the team and improve the common understanding of the product.

Testing isn’t done by a separate team using separate tools, it’s done by the team for the team, to deliver value to the customer.

So what about Testers?

So where to professional testers live in this world?  There is still a role for professional testers, but it’s not at the lowest level of development-testing, it’s a little higher up where having creative intelligent humans with a quality focus can provide the most value to the software value stream. Human testing isn’t especially useful for testing the normal flows, it’s useful for testing the “fringe cases” and doing exploratory testing.

Focus testing professionals on testing the end-to-end integration scenarios where many of the real problems tend to lie. Testers have a wealth of experience and knowledge of different test techniques that can be applied to address different quality risks, wasting that on simple stuff means they don’t have time to do the important stuff.


Process vs. Tools

What’s more important: Process or tools?

I’ve sat through numerous arguments with people trying to work out whether process or tools are more important to their software development effort. The arguments go along the lines of:

Process Geek: Tools are just the things you use to do the process, the value is in the process. Let’s do a process first roll out.

Tools Geek: Tools are the things people actually use. Process is academic and is constrained by the tool features anyway. Let’s do a tools based roll out.

Sadly both of these are real life quotes from people I’ve worked around in the past. I won’t embarrass them by naming them because they’re both really wrong. For me the real value is not in the tools or the process but in the people. Process and tools are something that you add if the people need them.

Minimal Tooling

For me the best planning tool is a whiteboard. The best architecture and design tool is also a whiteboard. I like to capture some info about what I’m doing and how close I am to achieving my goals, some really simple measures. The best tool for that is probably Excel, despite my love of Open Source. Other than that I need to manage some lists of things like requirements (stories/use cases whatever), defects and whatever other bits the team thinks are important. They might fit on a whiteboard but typically we want to have some attributes on them, be able to sort them and link them. The one downside of a white board is the lack of drag and drop. I’ve used a few different types of smart board and active touch screens but none of them have really given me the smooth experience I want.

The one bit of tooling I really need is my IDE + SCM + Continuous Integration.

Minimal Process

This is potentially a can of worms if I go into things like process kernels and ideological arguments about different agile approaches and practice collections. There’s no such thing as a process silver bullet so for me though it’s all about the team and being effective. Teams need to:

  • Have Clear Obligations
  • Have Clear Business Priorities
  • Seize their autonomy
  • Have the relevant technical skills and resource availability
  • Be active in actually doing their work (seems obvious but it’s something a lot of processes seem to hinder rather than help)
  • Demonstrate visible progress (via working software not a tracking gantt chart)
  • Continuously improve themselves (because no way of working is perfect)

This list based on the 7 Habits of Agile Teams by Roly Stimson

So why go more complicated?

Unfortunately life is rarely simple. Many teams work in complex environments where a number of factors might make them choose to need more than this kind of minimal set including things like physical distribution; audit, regulatory or security constraints; cross-team interaction in system of systems environments, more people etc.

When faced with increased complexity it might be appropriate to add more powerful tooling or development practices. The more people involved in a project the harder it is to herd all of those socially complex cats into working together.

This is why ALM suites exist ranging from plugging together some open source bits like git, SimpleGit, jenkins, agilefant, bugzilla etc. to commercial vendor products like IBM Rational RTC (which does agile planning, work item management, scm and build in one tool) and other IBM Rational Jazz tools – it’s worth noting that you can also get RTC for free for just 10 users or for academic/research use on jazzhub.

Alternatively there’s Microsoft TFS for source control, work item tracking etc. tying together existing Microsoft stuff like MS Office, Visual Studio and sharepoint if you’re a more Microsoft focussed development house.

Of course you can also mix and match this stuff to a greater or lesser degree with various integrations. No toolset is a silver bullet however and each have their own problems.

So what’s my point?

My point is that process and tools need to reinforce each other, that both should be selected based on the needs of the team and the environment the team has to operate in. In the process vs. tools argument both lose out to the people. People in a development organisation create value, they just use tools and process to do it.

What does “Agile at Scale” mean?

There’s a lot of talk in the process improvement industry about the meaning of “agile at scale”, devops, lean and agile for a number of reasons. One reason that I’ve seen in successful agile organisations is that as development maturity increases with true agile adoption bigger problems present themselves. This is the natural progression of the science of development, what used to be considered complex (like Object Orientation) is now normal, a commodity. Innovation in ways of working is happening at the organisational, cross-team, cross-product level.

For me agile at scale (I’ve got fed up of the quotes already) means a couple of different things:

  • Repeating agile successes embodied in a team across an organisation (scaling out?)
  • Applying agile thinking to cross-product projects
  • Applying agile and lean thinking to development organisations
  • Applying agile and lean thinking to high assurance environments like medical, security, financial, safety critical, audited, regulated businesses.

Agile and lean? Yep, both with lower case letters. I’m not particularly interested in ideological approaches to software development, I believe strongly in taking the best bits of whatever processes, techniques, practices etc. you find externally, mixing them up with internal practices and ways of doing things to develop simple, pragmatic approaches to ways of working. Both agile and lean schools of thought promote minimising unnecessary work, shorter delivery cycles and higher quality, continuously learning lessons and empirical decision making.

The agile manifesto gave us:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on
the right, we value the items on the left more.

Some great practices have evolved for applying agile and lean thinking like scrum, kanban etc. However all of the complex organisations I’ve worked with have found that there’s still space for more thinking in terms of how to run a software business, how to deal with big complex system-of-system problems, multiple competing stakeholder sets, programme and portfolio management etc. Not surprising really because the agile movement wasn’t about trying to do any of that stuff.

However organisations who are successful with agile transformations want to apply the successful open and honest philosophy behind the agile manifesto to other parts of their business as well as bigger and bigger projects and programmes because the results of “doing agile” (I promise I’ll stop with the quotes soon!) are so attractive when it’s done well, namely:

  • Shorter delivery cycles, higher quality
  • Deep engagement between customers and development teams leading to respect, collaboration and better morale
  • Quick identification of when things are starting to go wrong

Consider the following model, not uncommon amongst large organisations:

Diagram of nominal large inter-dependant organistion structure

This represents a typical software department or vertical section of a software department with a portfolio that provides the funding for work. Big portfolio’s are normally broken down into a number of programmes which in turn may levy high level requirements onto organisations (organisational sub-divisions that own multiple products) which may affect both legacy and new product development. Often within a vertical section of a business there will be many cross-dependencies in terms of requirements, technical dependencies etc. For many large businesses this picture is overly simplistic, indeed I’ve not included things like projects, component teams and a variety of business organisation constructs like product centres, feature teams etc.  So how do you apply agile and lean philosophy to this lot and more?

You can’t simply repeat the same practices recursively throughout an organisation to deal with larger scale complexity.  Imagine a chain of scrum-of-scrums, daily stand-ups at every level (at the same time, or staggered to allow representation up the chain?), sprint plans at programme level etc. What about if the business is regulated, audited, security focussed, high risk financial, safety critical, etc.

Ok, so what’s agile at scale then?

Agility at Scale is applying the spirit of agility and lean thinking if not the letter to these bigger problems. It’s about:

  • Valuing individuals and interactions, encouraging collaboration, reducing layers of communication over processes, tools and hierarchy
  • Valuing working software in the form of quality releases from short development cycles over comprehensive documentation, business analysis, enterprise architecture documentation
  • Valuing customer, business, developer and operations (see DevOps) collaboration over contract negotiation
  • Valuing good governance, transparency and honesty in progress, plans, costs and impediments over regular reporting
  • Valuing responding to change over following a plan at all levels of the business

(Borrowed from the Holistic Software Manifesto)

Agility at scale is focussed simply on reducing unnecessary bureaucracy, reducing time to market and improving value.

So how do you achieve it?

The application of:

Of course each of those (and more!) is a complex can of worms in itself. A lot of these higher scale practices are only just emerging from high maturity complex (post-)agile organisations but in time more of those things will turn into links.

A good example of  “Agile at Scale” in action is the Project Forum practice

As always this blog is  a stream of consciousness so please, let me know your opinion?

Advanced RTC SCM Streaming – Part 1 Development, Integration and Release Streams

Serious Gaming for education with complex layered metaphors

I’ve written a couple of blogs about metaphors. In Direct, Indirect and complex Metaphors I promised I’d post again on complex layered metaphors so here’s the post.

Complex Layered Metaphors

I consider a complex layered metaphor to be a collection of metaphors contained within another metaphor. Potentially this structure can be recursed so that you end up with with a stack of metaphors containing metaphors. There’s a number of reasons for doing this:

  • it’s fun
  • it provides a vehicle for delivering a lot of direct and indirect messages

Since it’s a little complex by it’s nature I’ll describe it in terms of a concrete example of some work I recently did with one of my clients. We wanted to get a number of messages out about awareness and the nature of the process improvement team in the organisation and services as well as specific messages in terms of the attractiveness of continuous integration, change set based scm (software configuration management), the use of a wide variety of collaborative tools.

To do all this in an innovative and creative way I designed a complex layered metaphor in terms of a Serious Game. The game itself was a treasure trail through a series of physical and intranet/web tool based locations.

Here’s the in-game metaphorical framework design:

Analogy Message(s)
[Top Level, contains everything else] The game is a complicated sequence through a diverse range of information sources and collaborative tools [Process Improvement Team] can make a path through the complexity of internal tools and technologies[Process Improvement Team] enables problem solvers to achieve their goals in innovative ways
The game brings together and links up a number of different areas of knowledge and tools (an information web made of new links)The game is rooted in the intranet which provides an information web of existing links.The game involves a clue that is a spider at the centre of a physical “web” which is actually the roof of a large virtual presentation area, literally a web of information. [Process Improvement Team] brings together disparate systems and knowledge into unified ways of working.There is a wide web of interconnected elements that are needed to complete our tasks.
Generally clues are meaningless out of context (e.g. just numbers, the name of an animal etc.)Information is meaningless without context, a broad understanding of the information architecture is necessary to understand the facts. Silo mentality in functional disciplines is working with information out of context. Cross-functional understanding and working, collaboration, is necessary to achieve goals.
Keep coming back to searching in [Internal Search Engine] for general knowledgeKeep coming back to searching in [Project Tool] for project information Much information is open and discoverable, knowing where and how to look is necessary. This is taught implicitly.
The name of the game [Quest] implies a search with a goal Doing the game is worthwhile and will result in finding something

The set of clues that made the game each led to each other and were themselves metaphors, normally containing metaphor restrictions to apply specific process improvement, service management and tooling messages indirectly. There were of course direct messages peppered throughout as well as well.

This provides three layers of metaphorical messages so far, the clues, the framework and the treasure trail game. On top of all of these was an activity sequence which started with a viral messaging campaign that leaked the information about the game in mysterious fashion. The discovery of the clue sequence, engagement in it’s elements (including 3 sequences held inside a virtual 3D environment like Second Life), and entry into the prize draw for a number of prizes ranging from the silly to the seriously cool.

As a result of the external activity sequence the players interacted with individuals in the Process Improvement Team through a variety of communication channels and even started eliciting help from them on a number of topics. Finally they had to physically come to the prize draw which further raised the profile of the Process Improvement Team and it’s individual members whilst delivering a message that the team was approachable and had answers.

This is just an example of how you can design layered complex metaphorical messaging campaigns. You don’t have to build a game structure for this kind of stuff.

Serious Gaming

Serious Gaming” is used to describe games designed for a primary purpose other than pure entertainment. Serious games are often used for educational, investigative and promotional purposes.

I’m a big fan of serious games and gamification as games can be excellent motivators. Gamification is the application of game concepts to solve problems and engage audiences/players. For an example of gamification in the wild check out Chore Wars which helps turn boring household chores into a game.

I play a game with myself to increase the hit count of my blog, for no other purpose that to increase the “points” I get 🙂 Indirectly that makes me ensure my blog is discoverable and I exercise good SEO practices to get good google rankings, oh yeah and post stuff that people want to read (I hope).

Learning through play is considered perfectly normal for children and yet is sometimes frowned up with adults.  As mentioned before, a serious game can be an excellent vehicle for metaphorical messaging as it can provide a framework of engagement and activity towards a goal. Make the activity a metaphor (like the quest example above seeking a goal by working through a web of complex tools and collaborative environments being a metaphor for the process improvement team enabling people to solve problems) and you’ve got a ready made layered complex metaphor.

If you want to run something like this here’s the set of game features I used to “gamify” the eductional treasure trail:

  • The treasure trail itself, they’re fun
  • Clues range from simple to complex to challenge players
  • Clues range from simple directions to multi-part referential clues that need piecing together (mini-puzzles)
  • Depending on the path taken through the game some clues can be referenced more than once in different contexts (i.e. the launch blog entry has a set of textual clues to the first location via encoded IP address, contains a red herring reference embedded in decorated characters in the text and also contains the final code embedded in the task). Discovery of hidden information, especially hidden patterns or elegance is psychologically rewarding for humans.
  • There are a number of “red herrings” in the game, adding another mini-game in terms of find them
  • 3D virtual environments imply game activity and lead to fun construction and interaction activity. There was also a rudimentary tic-tac-toe game in-world.
  • Clues are written in stylised humorous game-like language
  • Competition for prizes (open prize draw for all finishers who submit the correct code word, minor silly prizes for fastest and slowest completion, minor prize for funniest feedback comment)
  • The winning of virtual badges by completing the game that people could add to their profile
  • Early viral messaging campaign to make the game seem “subversive” initially

Feedback was overwhelmingly positive, in fact some of the most positive feedback I’ve ever seen, and we estimated we completed several hundred hours of high engagement practitioner training for only 4 days effort (game design and running) + the cost of the prizes.

All work should be like this, creativity should be the norm, not the exception.

This blog is part of a series on Holistic Communication: The linguistics of business change. Introduction, ethics and table of contents is all in the first post.

%d bloggers like this: