Mike MacDonagh's Blog

Somewhere in the overlap between software development, process improvement and psychology

Tag Archives: IBM

Process vs. Tools

What’s more important: Process or tools?

I’ve sat through numerous arguments with people trying to work out whether process or tools are more important to their software development effort. The arguments go along the lines of:

Process Geek: Tools are just the things you use to do the process, the value is in the process. Let’s do a process first roll out.

Tools Geek: Tools are the things people actually use. Process is academic and is constrained by the tool features anyway. Let’s do a tools based roll out.

Sadly both of these are real life quotes from people I’ve worked around in the past. I won’t embarrass them by naming them because they’re both really wrong. For me the real value is not in the tools or the process but in the people. Process and tools are something that you add if the people need them.

Minimal Tooling

For me the best planning tool is a whiteboard. The best architecture and design tool is also a whiteboard. I like to capture some info about what I’m doing and how close I am to achieving my goals, some really simple measures. The best tool for that is probably Excel, despite my love of Open Source. Other than that I need to manage some lists of things like requirements (stories/use cases whatever), defects and whatever other bits the team thinks are important. They might fit on a whiteboard but typically we want to have some attributes on them, be able to sort them and link them. The one downside of a white board is the lack of drag and drop. I’ve used a few different types of smart board and active touch screens but none of them have really given me the smooth experience I want.

The one bit of tooling I really need is my IDE + SCM + Continuous Integration.

Minimal Process

This is potentially a can of worms if I go into things like process kernels and ideological arguments about different agile approaches and practice collections. There’s no such thing as a process silver bullet so for me though it’s all about the team and being effective. Teams need to:

  • Have Clear Obligations
  • Have Clear Business Priorities
  • Seize their autonomy
  • Have the relevant technical skills and resource availability
  • Be active in actually doing their work (seems obvious but it’s something a lot of processes seem to hinder rather than help)
  • Demonstrate visible progress (via working software not a tracking gantt chart)
  • Continuously improve themselves (because no way of working is perfect)

This list based on the 7 Habits of Agile Teams by Roly Stimson

So why go more complicated?

Unfortunately life is rarely simple. Many teams work in complex environments where a number of factors might make them choose to need more than this kind of minimal set including things like physical distribution; audit, regulatory or security constraints; cross-team interaction in system of systems environments, more people etc.

When faced with increased complexity it might be appropriate to add more powerful tooling or development practices. The more people involved in a project the harder it is to herd all of those socially complex cats into working together.

This is why ALM suites exist ranging from plugging together some open source bits like git, SimpleGit, jenkins, agilefant, bugzilla etc. to commercial vendor products like IBM Rational RTC (which does agile planning, work item management, scm and build in one tool) and other IBM Rational Jazz tools – it’s worth noting that you can also get RTC for free for just 10 users or for academic/research use on jazzhub.

Alternatively there’s Microsoft TFS for source control, work item tracking etc. tying together existing Microsoft stuff like MS Office, Visual Studio and sharepoint if you’re a more Microsoft focussed development house.

Of course you can also mix and match this stuff to a greater or lesser degree with various integrations. No toolset is a silver bullet however and each have their own problems.

So what’s my point?

My point is that process and tools need to reinforce each other, that both should be selected based on the needs of the team and the environment the team has to operate in. In the process vs. tools argument both lose out to the people. People in a development organisation create value, they just use tools and process to do it.

How to do maven releases with Jazz SCM

I’ve previously blogged on structuring maven multi-module projects to work well with m2, eclipse and Jazz SCM. In this post I’m going to write about how to do releases because it’s not as straight forward as you might hope and expect. This blog is explicitly about using maven 2.2 and IBM Rational Team Concert 3.0.1.2 although it will apply to various other previous versions. Later versions of RTC should (I hope) alleviate some of the problems and the Jazz SCM Maaven plugin is being actively work on.

Overview

During development or maven projects we start with a label such as 1.0.0-SNAPSHOT which reflects the current development build. Over time the development snapshot might be occasionally pushed to a maven repository as a “snapshot release”  if it’s roughly stable so that dependant teams can get the dev version if they really want it. In time development gets stable enough to release the definitive 1.0.0 so the -SNAPSHOT is stripped from the label and the version is pushed to the repository as a proper “release” for other teams to depend on. At this point a new number is selected and development continues on the new label such as 1.0.1-SNAPSHOT which might be occasionally pushed etc.

Diagram showing how releases and version numbers work with maven

The problem

This process works pretty smoothly in a lot of contexts as the release is done by issuing a command line as simple as mvn release and maven’s smart enough to work out the rest. The problem comes in the context of Jazz SCM for a couple of reasons:

  1. The Jazz Maven SCM plugin is still in early development so doesn’t do everything well and some things not at all
  2. The Jazz Maven SCM plugin incompatible with the maven release plugin because it requires the pom scm tags to be written in a way that the release plugin can’t read
  3. The jazz scm plugin insists on outputting your password in cleartext all over the logs so you can’t do it as part of an RTC build without exposing the build user password (which depending on your organisation might not matter or could be a huge deal)
  4. Both maven and rtc use the word snapshot for different purposes and other terms like “baseline” and “tag” are synonymous

The solution

Write a little script to work around the problems with the following requirements:

  • All published version numbers in the maven repository must be reflected by RTC scm snapshots
  • The process of releasing is simple to use

To make this work I’m actually dumping the use of the Jazz Maven SCM plugin as it’s not yet developed enough to be useful. Instead since every build happens in RTC occurs in a snapshot here’s the process:

  1. Green Build in RTC (optionally promoted to RTC Release) with a known snapshot name e.g. v1.0.1 release
  2. Integrator/Releaser/Committer/Nominated dev runs custom mvn-release script on their local machine
  3. Sits back and watches, feels good

Here’s the pseudo-code for how to do it all. One of my clients is developing this currently and I’m reasonably optimistic that they might open source (or allow me to) the full script. However if you’ve got a bash monkey working for you it shouldn’t be hard to develop this up pretty quickly.

Currently this process needs some parameters which could be passed in or the user could be prompted. I prefer to pass in and prompt so that the script can be invoked from a remote process (like a build server) passing in properties or can be interactive if a user invokes it. Parameters required at least somewhere along the way are:

  • rtc user name and password
  • is this a snapshot release or proper release
  • rtc build snapshot
  • new version number to base label (if doing a proper release)

CURRENT_VERSION = getCurrentVersion(pom.xml) //1.0.0-SNAPSHOT

//If just doing a basic snapshot release
scm create snapshot "CURRENT_VERSION"
scm checkin & deliver
mvn deploy

//If doing a full release
PROJECT_NAME = getCurrentProjectName() //basename of current directory
NEW_WKS_NAME = create new rtc workspace in /tmp/rtc/workspace/RTC_SNAPSHOT_NAME/
load temp workspace with contents of RTC_SNAPSHOT_NAME
cd /tmp/rtc/workspace/RTC_SNAPSHOT_NAME/PROJECT_NAME
//Generate Release
RELEASE_VERSION = StripSnapshot(CURRENT_VERSION) //1.0.0
mvn versions:set -DnewVersion="RELEASE_VERSION"
scm create snapshot "RELEASE_VERSION Release"
scm checkin and deliver
mvn deploy

//Generate new development starting point
NEW_VERSION = AskUser(); //1.0.1-SNAPSHOT
mvn versions:set -DnewVersion="NEW_VERSION"
scm create snapshot "NEW_VERSION Initial baseline"
scm checkin and deliver
//don't do a mvn deploy as that would be the user choosing to run this script choosing the snapshot-release option

//Tidy up
scm delete workspace NEW_WKS_NAME

The advantage of running this script creating a temporary local workspace and running from there is that you can do this from the command line without closing eclipse (or invoke from Eclipse as an external tool configuration). Also it isolates the release process from any possible local changes in the /Releaser/Committer/Nominated dev’s workspace.

Until the Maven SCM plugin comes of age I hope this helps.

Advanced RTC SCM Streaming – Part 3 Component reuse

Basic usage of RTC Jazz SCM is easy enough but there’s what about more advanced usage? Using streams to isolate development from integration? Cross-project component reuse? Parallel branched development? Integration Streams aligned to Definitions of Done? This is the guide for that stuff.

This blog is in three parts:

  1. Development, Integration and Release Streams
  2. Streams for parallel branched development
  3. Streams for component reuse

Part 1 also covers the introduction and overall conclusions

Working with components across multiple RTC projects can present a bit of a challenge. It’s simple enough to create a stream with any baseline of any component in your repository but dealing with changes to those common components is not simple or obvious, especially if you need different visibility controls and access rights for different teams on different components.

Read more of this post

Advanced RTC SCM Streaming – Part 2 Parallel branched development

Basic usage of RTC Jazz SCM is easy enough but there’s what about more advanced usage? Using streams to isolate development from integration? Cross-project component reuse? Parallel branched development? Integration Streams aligned to Definitions of Done? This is the guide for that stuff.

This blog is in three parts:

  1. Development, Integration and Release Streams
  2. Streams for parallel branched development
  3. Streams for component reuse

Part 1 also covers the introduction and overall conclusions

Change set based SCM systems like git (SimpleGit)and RTC do away with the need for a lot of branching. You don’t need to branch to get developer isolation, to gather file changes together to understand them as an atomic set of changes related to a bit of work or to manage concurrent development of the same files on a small scale. You get all of that for free by using change sets. Developers are always in their own private personal branch separated from the rest of the team but unified by a common stream or repository. However sometimes it’s still appropriate to branch, this blog covers some of the situations when it’s appropriate and how to do it effectively in RTC Jazz SCM.

Read more of this post

Advanced RTC SCM Streaming – Part 1 Development, Integration and Release Streams

Working with Maven multi-module projects in RTC Jazz SCM with m2

I’ve been working with Maven projects with m2 for a while in RTC Jazz SCM so thought I’d post some notes. For reference this post covers using RTC v3.x things may change in the future. If you’re not using m2 then I’d suggest you read this blog by Philippe Krief.

First of all it’s important to understand that RTC v3 has limited support for Maven. You can setup a build definition which can call a maven goal (such as a local install) easily enough but that’s about it. The RTC build mechanism is basically a cron job with a nice web (like Hudson/Jenkins without the plugins) and Eclipse front end, it doesn’t have a deep understanding of Maven.

The whole point of Maven over Ant is that it does a lot by convention rather than specification, however in most advanced Maven project I’ve seen pom files are as big as equivalent Ant + Ivy files. Regardless I still think Maven with the excellent Sonatype Nexus is awesome (although I also like the structured bash/batch like nature of Ant as well).

Most serious Maven projects are multi-module. This is because component based development is a good thing (shameless employer plug, but it’s true, it is a good thing). Most (tending to all) multi-module projects have the following structure where “O” is a folder and “-” is a file, considering a hypothetical ProductX:

– pom.xml (the parent pom for the master/uber/product build, lists child modules)

– various files at the root of ProductX

O ModuleA

– pom.xml (for ModuleA)

– various ModuleA root files

O various ModuleA folders of stuff (such as src)

O Module B

– pom.xml (for ModuleB)

– various ModuleB root files

O various ModuleB folders of stuff

If you want to work with this stuff in RTC Jazz SCM then you need to balance the constraints of Maven m2, RTC and Eclipse. If you’re considering doing this then you’ll probably be considering moving existing Maven projects to RTC, in which case you may find a previous blog on a manual migration script useful. The only edit to this I make for Maven is to separately import the parent from the children if flattening the structure (although in most cases I wouldn’t recommend flattening).

The constraints playing against each other are:

  • Maven and plugins expect sub-modules to be in child directories as in the tree above
  • RTC doesn’t allow both an Eclipse project and root files to be at the root of a component
  • The simple case of Eclipse and RTC SCM interacting during a load from the server restricts Eclipse to only seeing projects which are represented as folders at the root of the local Eclipse workspace containing .project files.
  • Eclipse doesn’t really support parent and child projects.

I’ve found there are a number of approaches, none are perfect:

1. Flatten the structure (my opinion the worst option)

You can edit the parent pom.xml to reference the sub-modules as ../ModuleA etc. although the Maven release plugin (amongst others) don’t play well with this. Also, and somewhat fundamentally, you shouldn’t have to restructure your code based on your SCM tool.

2. Only load the relevant bits (my opinion is it’s sometimes ok)

You could just load the child modules you’re currently working on and then unload (remember to delete local files or you hit a bug) and then reload the parent when you want to do a “big” parent build. Frankly this is unacceptable from a developers perspective and will fail anyway if there’s a dependency on the parent from the children. Although in theory this may be frowned upon it’s quite common and so needs supporting by the scm tool.

3. Load then import, with auto-refresh (my preferred option)

This one is going to be more than one paragraph….

Prediction fulfilled ;p Anyway… Currently your developers may be used to a scm system where they load/copy/access files from the scm system on the local filesystem and then do an Eclipse -> Import Existing Maven projects to get all of the various goodies.

If you import the multi-module project as above but with a root folder (as an eclipse project) so it has the following structure:

O Product X

.project

– pom.xml modules)

– various files at the root of ProductX

O ModuleA

– pom.xml (for ModuleA)

– various ModuleA root files

O various ModuleA folders of stuff

O Module B

– pom.xml (for ModuleB)

– various ModuleB root files

O various ModuleB folders of stuff

Then when you do a load from Jazz SCM you’ll get in the Eclipse Project Explorer (or equivalent) ProductX correctly detected as a Maven project (or an Eclipse project with a Maven nature for the pedantic amongst you).

You can then do a normal Eclipse -> Import Existing Maven Projects based on your workspace/ProductX/… (taking care not to select “Copy projects into workspace”) You’ll then get in Project Explorer (or whatever):

ProductX (info about rtc scm links)

ModuleA

ModuleB

In this view ModuleA and ModuleB exist in two places: as projects referenced in the workspace but apparently outside of RTC SCM control; as a folder structure under ProductX/ModuleA and ProductX/ModuleB.

This has the following advantages:

  • Cross-project dependencies resolve
  • You can run the parent/uber pom goals and the child pom goals without reloading anything
  • You can shift focus between the child-modules and parent structure without reloading anything

If you edit files in ModuleA and ModuleB under the folder hierarchy of ProductX then Pending Changes will keep track in the normal way but if you edit a file under the apparent “root level” ModuleA and ModuleB then Pending Changes won’t unless you do a “deep refresh”.

Alternatively just go to Eclipse -> Window -> Preferences ->  General -> Workspaces -> Refresh automatically (er… I think, I’m doing this from memory, hence the lack of screenshots). You can then add “edit files tracked by Pending Changes at either Product or child module level” to the advantages above.

So this option sounds great so far, what are the downsides?

This solution maintains the hierarchical structure between ProductX and it’s child modules and as such constrains everything from ProductX downwards to be in the same Jazz SCM component as eclipse projects can only be seen (without far too much fiddling) if they’re root folders in components. This may not seem like such a big deal but components are the lowest level of baseline (or label) in Jazz SCM. In a tightly coupled cohesive single architecture this might not be a problem but if you want to reuse any of these modules (at a code level) or have a separate ownership it will be a problem. It’s also very common to have dependencies from child modules to parent projects (which may initially seem to resolve if they’re in the local ~/.m2 repository).

Having said that source code dependencies between components lead to brittle architectures, binary dependencies are far superior, and made much easier by the use of things like Ivy/Maven and Nexus.

Other considerations

The point above above binary vs. code dependencies can be argued until the cows come home and go out again (an extended metaphor), especially if you consider project/product/team/release boundaries. I’m not going to pretend to answer those with an academic silver bullet here, the important thing I hope you take away from this blog is that there are pros and cons to all of these approaches, balancing those with the needs of your team/architecture/project/organisation are not easy. Shameless plug: Give me a call if you need help

Eclipse refactoring isn’t dealt with well by Jazz SCM, especially between Eclipse projects in and “out” of source control (advocated by option 3 above). Expect a lot of “remove/add” combinations if you are doing anything more than a trivial set of file changes. This is also a problem for the ClearCase <-> RTC automatic synchroniser developed by the excellent Samecs however I know that I, others and Samecs are pushing for improvements on this from IBM.

This post has been very scm focussed but what about build and release? The RTC build engine understands maven to a limited degree and can easily invoke maven builds for information on integating the maven release process see How to do maven releases with Jazz SCM

RTC: How to write cross project queries

Ok, so I’ve previously said that you can’t write cross-project work item queries in IBM Rational Team Concert but that’s not entirely true… You can, but it’s a bit “hacky” and isn’t very supportable. In Eclipse go to:

Team Artifacts -> My Team Areas node, select a user -> right click -> Show Recent Work

This seems to create a work item query that has the user as a top level scoping element rather than the Project Area, you can even add the Project Area column to the result set 🙂 You can edit this query in the normal way but it seems to save itself to a project area selected by rolling a dice and there’s no way to control which one as far as I can see. Also, remember that it’s top level scope is the user that the query was generated from, that may have interesting effects.

Simples.

Don’t forget to demand proper cross-project querying from IBM on jazz.net

Edit: 10/11/2014 – to choose which project area the hacky query ends up in simply disconnect all of the other projects areas (and restart eclipse).

First Look: CLM 2011 RTC + RRC + RQM 3.0.1 review and screenshots

What is it?

CLM 2011 is a suite of tools making up IBM Rational Team Concert, Rational Requirements Composer and Rational Quality Manager properly integrated for what is arguably the first time. In my opinion this release is the closest to my interpretation of the original Jazz vision that IBM have delivered yet. It’s not perfect but I like the direction it’s heading in.

It’s taken me a while to write this, mainly because I’ve been very busy but also because it seemed to take some deep magic to get all of this stuff installed and running properly. But I’ve done it a couple of times now so I thought I’d post some screenshots and comments. Of course if you look back on my posts of the original RTC review you’ll see that life is harder these days already. With RTC 1 and 2 you could just download a zip, extract, run a script and you were off and away. Installation isn’t too bad… it’s not great and simple but this is a big complex suite of tools so I don’t really expect it to be trivial.

Lifecycle Project Administration

To start with, the management of projects across these separate tools has been significantly improved by the introduction of Lifecycle Project Administration which allows you to create an integrated cross tool project in one step and simply manage members and roles across them. This is a big step forward although there are still some problems in that it’s not easy to do these kind of things at scale. For example if I want to add 3 people with a set of roles to 40 projects I’m going to be doing a lot of manual clicking. In fact generally it’s not so easy to deal with project areas at scale in the Jazz toolset and that hasn’t significantly improved yet although Lifecycle Project Administraton is an important step in that direction.

Creating a new Lifecycle Project

Creating a new Lifecycle Project

A Lifecycle Project

A Lifecycle Project

Editing roles accross tool project areas

Editing roles accross tool project areas

I’m not a big fan of the navigation between project areas (linked or unlinked) as the user needs to understand the architectural relationship between personal dashboards, Change and Configuration Management (which has the RTC bits like plans, work items, source control, builds etc.), Quality Management (which has the RQM bits like test plans, cases, scripts etc.) and Requirements Management (which has the RRC bits like diagrams, UI sketches, docs, requirements etc.) to navigate the stuff in their single project. I think it’s a mistake to build the UI navigation around the architecture, I would prefer to see a unified project interface and navigation structure with the extra products adding to the project mass like aspects on a foundation. As before this becomes even more of an issue when you scale up to hundreds of projects. Incidentally, aspect orientation is how we apply practices to process kernels while still providing a seamless user experience of composed practices.

Cross (linked) project navigation

Cross (linked) project navigation

So although the three products are closer together than ever before, sharing a UI and a common architecture they still are separate products and that’s clear in terms of navigation, UI differences  and linking items between them. This is a shame for many reasons but one of the most important is that it’s still providing niche tools for separate roles, building walls between people in development teams and making cross-functional teams harder to create as individuals have to learn specific skills. These differences are not a strength, they make the whole game of software development harder.

To be fair though I’m yearning for an ideal perfect solution, CLM 2011 isn’t that idealised perfect solution but it’s a lot closer to it than anything else I’ve used!

Let’s start with IBM Rational Requirements Composer

RRC 3.0.1 is a totally different beast than RRC 1 (see my original 2008 review here) and shouldn’t be thought of in the same light as RRC1. The old version was eclipse only, badly integrated, dependent on IE and full of bugs – this version is entirely web based, deeply integrated into the Jazz platform and not dependent on IE! As for bugs, I don’t know yet but it’s actually usable which v1 wasn’t!

What it does:

  • Tracks requirements of different types with different attributes and links
  • Web based diagram editing (via browser plugin for FireFox or MSIE ), although someone used Microsoft CAB technology inside the FireFox xpi so don’t expect to do any diagram editing from linux or a mac 😦
  • Ability to produce use case diagrams, UI prototypes, UI storyboards

For me this is the first time RRC has lived up to some of my early hopes for it. I see this as a replacement for ReqPro, and indeed a replacement for DOORS in time.

Creating Requirements Artifacts

Creating Requirements Artifacts

Linking a requirement to a new implementation story

Linking a requirement to a new implementation story

Unfortunately I only use linux at home so I couldn’t take screenshots of the RRC web editor, even in a virtual machine running windows on an explicitly supported browser I can’t get the plugin to work. When I’ve used it my office environment I’ve really quite liked it though, although it’s got quite a lot of basic usability problems. I’m looking forward to when it’s more mature.

There is also a dependency/traceability tree viewer but that’s a separate download at the moment.

Next Implement something in IBM Rational Team Concert

RTC hasn’t changed that much from a user perspective, it’s still a great tool for tracking work items, making atomic multi-file change sets in parallel with continuous integration and build management. I’ve been using RTC in anger for 3 years now and still like it and think it can help drive agile benefits in development teams. Granted it’s got some issues like any tool, enterprise scalability issues and annoyances the biggest of which is the lack of cross-project queries which is practically unforgivable from my perspective. See this work item on jazz.net and demand cross-project queries from IBM!

With that said, working in Eclipse from workitems, fully using Jazz SCM and continuous build is awesome.

Edit a work item in RTC

Edit a work item in RTC

Sprint plan in RTC

Sprint Plan in RTC

Making code changes in RTC Eclipse

Making code changes in RTC Eclipse

Looking at build results in Eclipse

Looking at build results in Eclipse

Looking at Build results in the web

Looking at Build results in the web

Then Test via IBM Rational Quality Manager

I’ll admit it, I’m not a cross-functional ideal. I’m a typical developer, I’m not a tester, nor am I a test manager so I’ll leave the finer points of testing theory to others. However I do have some things to say about RQM from a cohesive logical process perspective.

In this example I’ve created a Test Case based on the pizza requirement I entered in RRC (in exactly the same way that I created the implementation story). At this point frankly I’m a little disappointed because my Test Case has been created and is linked to the requirement (good) but it has no knowledge of the implementation story that I also created from RRC.

Test Case in RQM

Test Case in RQM

Test Case link back to requirement

Test Case link back to requirement

No pre-existing link to development items

No pre-existing link to development items

The Missing Link

For me this is the missing link. I realise it’s not as a simple as their always being a related triangle of 1-1 associations between these different types of artifacts, but the link between the test and the development items should at least be suggested as the odds are fairly strong that if a requirement is developed by an bit of code the test for the requirement is likely to need to test the aforementioned bit of code. Obviously I can manually create this link but that’s not the point.

In fact this is symptomatic of the fact that these CLM tools are still separate and then integrated. There is a separation between requirements in RRC, development plan items and tasks in RTC and test plan items and tests in RQM. I have to create work items /artifacts  in each tool to represent these different things and link them all together. Which is not really what I want to do.

I don’t want to spend a lot of my time creating at least 2 items for every requirement in my product (1 dev story+tasks and 1 test case+scripts).

What I want to do is elaborate a set of requirements in a simplistic user friendly UI with nice diagrams (RRC can do this) then view that candidate backlog immediately in my planning tool and further refine it – not copy it (even in bulk) to RTC but to actually manipulate the same work items, albeit a different aspect of them, in RTC. I want testing to be an inherent part of development with quality management and assurance being just another aspect of the system. Every requirement should have quality dimensions and be a test case although I might want to add more of course.

I want to have requirements with dev tasks and tests hanging off them.

Basically I want to define something that needs to be done, plan it, do it and test it all as one high level thing. I might drop off loads of tasks, different types of tests, supporting sketches etc. but I want a holistic understanding of a development item with the ability to project different views to different types of stakeholders (customer requirements views, test professional views, management views, development views).

Some conclusions

CLM2011 is a serious step in the right direction from IBM. In my opinion Jazz always has been but it’s been suffering from too many silly little problems and a lack of meaningful deep integration (ironic considering the original mission). If I had a magic wand I would do the following feats of greatness to the Jazz platform which I believe are all necessary to turn what is a good suite of tools into a true killer software development environment.

  • all jazz tools to apply new aspects to projects rather than creating seperate project areas which then need linking and administration via LPA
  • all artifacts and workitems to be viewable as native items in any tool interface
  • all artifacts and workitems created by all three tools (and other future ones) be instantiated and linked with truly consistent UI, all taggable and commentable
  • all artifacts, scm/build elements and work items queryable via a single query engine (across linked and un-linked project areas)
  • the ability to self serve projects (without granting massive impractical rights), finer grained security and permissions control
  • parent-child change sets
  • smoother flow of work items between tool aspects rather than creating new ones with links between them
  • make RRC diagram editing work on linux – things that only work on Windows are not enterprise deployable, and if they’re browser based and only work on windows someone needs shouting at. Even a much maligned school boy shouldn’t be making this error
  • a decent reporting solution and technology

Finally , being able to capture and elaborate requirements, plan iteratively, develop code, continuously integrate with unit tests, system test and manage test plans (amongst other things!) in what feels close to a single tool is extremely powerful. If only navigation wasn’t architecturally focused this strength would be much stronger and be a killer feature for IBM and Jazz.

If they sort out cross-project querying.

RTC: Structural patterns for Project Areas vs. Team Areas

How big should a project area be? What’s the relationship between Project Areas and Team Areas is an interesting problem and for new users the relationship can be a little hard to understand. This post describes the relationship between them and some experiences of using them.


I’m going to write a series of posts on top tips based on experience and real projects of using IBM RTC for Disciplined Agility. It’s a bit of a brain dump, but here goes… they’re all tagged as rtc_tips if you want to read the whole set :


Simply put a Project area is for work that will share the same work items, workflows, visibility (as of v2, v3 improves on this). Team areas are areas inside a Project Area that you can use to scope plans, burndown charts, dashboards etc. Initially therefore it might sound sensible to create quite large Project Areas and then create Team Areas inside the large area for each project/endeavour (what I call Model C) but there are some problems with this idea:

  • it’s not easy for users to create new team areas with all the associated things that need to go with them (timeline, categories, dashboard etc. – the team dashboard templating is rather limited so you can’t easily parameterise queries on a template)
  • people with rights to a project area can see all work items in that area (true for v2 but this has been somewhat improved in v3 allowing you to set 1 arbitrary project area to control the access to a work item regardless of it’s “parent” area)
  • once you have more than 2 teams and more than 2 timeline the navigation of current plans and understanding how to scope plans/charts etc. gets too messy for the average user. Indeed every real life team I’ve worked with that has used multiple timelines & multiple team areas in a project has after a while decided to split them up into separate project areas.

These factors have pushed me to always create a Project Area per project/endeavour (which I call Model A) which gives you better access control/visibility, permissions control, well templated project dashboards & navigability. However there are some downsides:

  • You can’t write cross-project queries. Or at least the capability is severely limited even if you exploit bugs in the querying engine. It is possible to put viewlets from multiple projects side by side on a dashboard but it’s not possible to do things such as “Show me the top risks by priority for these 5 projects” or more importantly “Show me all my tasks by priority from all of the projects I’m a member of” – see Jazz.net work item 94575 for more on this.
  • As a result it’s not so easy for an individual to work across a number of projects although cross project dependencies and traceability queries are just about ok (in Eclipse if you show relationships, but not really in the web)
  • Once you get a large number of projects on your server it becomes a little annoying that you can’t manage them other than as a flat list (sort by “My projects” then “All projects”)

I’ve come to think of these different options for structuring RTC on a scale of simplicity to complexity and factors that push you one way or another:

Structural choices for RTC configuration

Model A says [0..]1 Team Areas because although it’s possible to configure RTC to only have 0 team areas and make everything scoped to the project area I’ve found that to be a bit buggy, especially in templated charts on dashboards. So the simplest model I recommend is 1 Project Area with 1 Team Area. You can do quite a lot with Model A by using categories well, and having views of plans by category with different category owners. I’ve used this model with teams from 3 to 50! The best advantage of this model is that for any time period there’s only 2 plans, the overall Product Backlog and the current Sprint Plan. (Maybe a release plan, but that can be optional as it’s just a subset of the Product Backlog).

For non-software this Model still works but depending on the required simplicity I would either go for no plans at all, or a single plan that is a mix of the Product Backlog and Sprint plan views.

Model B offers a bit more complexity in that there are multiple teams, seperated by categories but sharing a timeline. The advantages of this model are that you can manage multiple teams in seperate backlogs and plans but you can also create backlogs/plans that span all of the teams giving you individual views and an overall view. The downside is that this can start getting quite complex in terms of navigation as for any time period there can be a multitude of plans, not including release plans (as optional as before) that leads to 2 * (no. teams +1) plans if you’ve got 3 sub-teams and you’re on 2 week sprints that can be 8 plans for the next two weeks!

Model C offers fully seperated teams in their own timelines and categories. This is a bit like Model B but they can be on their own rhythms. In fact in this model the My Work box in Eclipse can’t quite cope with the multiple timelines so users really are quite seperated, to the extent that they may as well be in different projects – so I say they should be. It’s better to have multiple Model A’s than a single Model C.

Obviously this is all generalised stuff based on the projects I’ve been involved with and other factors would affect the decision, notably SCM considerations over how component source needs to be dealt with across teams. Having multiple somewhat seperated teams, on their own timelines editing the same code base might make you want to adopt Model C but if you’re in that situation you’ve got bigger problems than navigating a set of complex plans! In fact to unpick the problem I would still recommend multiple Model A’s with their own dev streams, then deal with integration from an requirements, test and SCM perspective in a master project, which is where your Scrum of Scrum behaviours can be implemented.

RTC: What should and shouldn’t go on the backlog?

I’m going to write a series of posts on top tips based on experience and real projects of using IBM RTC for Disciplined Agility. It’s a bit of a brain dump, but here goes… they’re all tagged as rtc_tips if you want to read the whole set 🙂

So theoretically anything that the product owner wants goes on the backlog, but in RTC we tend to have other work item types like lessons learned, risks, etc. and it’s not always clear what should and shouldn’t be on the backlog.

I recommend using work item types such as:
Story – sized by points, used for Product Owner requirements
Objective – something you’ve got to do that will decrease you’re velocity (see below)
Defect – for bugs
Change Request – for changes

All of these I would configure as top level work items that I would expect as parents on the backlog. I use Objectives in a number of ways to deal with things that will reduce velocity (such as get a test environment ready). Of course it’s possible to phrase these as stories but that starts getting a bit silly, in a tool like RTC it’s far easier to read “Get Test Environment ready” rather than “As a customer I want the team to get a test environment ready so that I can have testing done without affecting the live system in a bad way”. I use them to act as a parent for work to do something such as mitigate a risk, improve team ways of working and other overhead activities. This means that overhead is explicit and honest, the Product Owner can see how much there is, what it is and discuss with the team if any of it can be reduced to improve velocity. “Hiding” these things as stories feels dishonest.

Naturally below all of these I’d then have task breakdown with time estimates. Tasks don’t belong on the Product Backlog but do exist on Sprint Plans.

I also configure RTC to manage a number of other work item types depending on the needs of the team:
Risk – no one ever delivered an important project successfully without managing risks
Impediments – raised at daily standups they should be resolved as quickly as possible, if that’s going to take longer than a morning then track them as work items, in distributed teams record them straight away.
Lessons Learned Capture them as you go
Milestone Not on the backlog but with dependencies on to Stories and Objectives which are you can then query for Milestones dependant on the current iteration, sorted by date, showing depends relationships. The team can then use high level milestones to communicate to external stakeholders in terms of progress towards major releases. A backlog full of detail is too complicated to give a headline summary, using Milestones is a good work around. I was interested to see this make it into RTC v3 formal project template as I’ve been doing it for a while in RTC v2. Milestones are especially useful as a point of integration between the dev project in RTC and higher level portfolio plans if you’re in that kind of environment.

So none of these latter work item types I would have on the backlog. I might create related objectives or stories to do something relevant to them but I wouldn’t directly put them on the backlog. For example a risk might have a mitigation strategy (objective) which involves some work (tasks) but once that objective has been done the risk may still exist. These things are all useful to manage and view separately from the detail and complexity of the backlog – also they just add noise when it comes to prioritising and gardening a detailed backlog.

Of course I would never follow these “rules” when they’re not practical. If there’s a risk that needs a simply investigatory task then I wouldn’t create an intermediary objective just to keep hierarchical rules perfect – that way leads to madness and the process police.

%d bloggers like this: