Mike MacDonagh's Blog

Somewhere in the overlap between software development, process improvement and psychology

Advanced RTC SCM Streaming – Part 1 Development, Integration and Release Streams

Basic usage of RTC Jazz SCM is easy enough but there’s what about more advanced usage? Using streams to isolate development from integration? Cross-project component reuse? Parallel branched development? Integration Streams aligned to Definitions of Done? This is the guide for that stuff.

This blog is in three parts:

  1. Development, Integration and Release Streams
  2. Streams for parallel branched development
  3. Streams for component reuse

Part 1 also covers the introduction and overall conclusions

Streams, components, builds and releases

First it’s important to have a common understanding of some of the terminology. In Jazz SCM code lives in components which have versions travelling along time in streams. So for a project a stream is the shared area code is shared to and got from, sub-divided into components.

A version of a component might be in several streams allowing those versions to be compared and changes to flow from one stream to another (yes really, you used to not be able to do this but you can now as of 3.0.1.something, I think maybe a 2 on the end).

Users interact with the SCM system using a workspace. Changes are made locally in a workspace, checked-in to the server (still private) and delivered to a stream to commit them to the shared area so other users will see the changes.

It’s a good SCM practice to keep your change sets small as it makes them easier to merge in the event of conflict. That means change sets will be delivered frequently and the stream will always be churning with changes.

Hopefully you’ve using a Continuous Integration practice and as changes come in builds are happening every few minutes or as appropriate. Of course you can do local builds to ensure you’ve not broken the build, and in Jazz SCM you can even run a “personal build” which is a build using your workspace and local changes but run on the build server. That way you can find out if you broke the build before you broke the build 🙂

This is all well and good for demo projects but real projects sometimes have huge long builds and maybe even occasionally corner cutting developers. The reality of life is that a development stream will and should churn quickly, sometimes the build may even break. The contents of the development stream aren’t always ideal for external testing, customer review, release etc. etc. And even when they are sometimes the point in time appropriate for release is hard to define amongst the constant churn of development. This is where multiple streams can come in to separate concerns 🙂

Simple SCM stream

So change sets happen all the time, triggering builds which happen all the time and should almost always be green. Every now and again (hopefully not too infrequently) a “release” is made which is appropriate for moving to other teams/customers/stakeholders for whatever purpose. More on builds and releases

Using streams to separate development, integration and release

Streams provide an excellent mechanism for isolating the churn of development from release. You can set up streams for development, as described above which flow to an Integration stream. The integration stream will then change relatively slowly (unless fed by multiple development streams when working with systems of systems or multiple component teams) providing a home for integration activity and higher level testing. Once this activity is complete a “release” made from this stream can promote a snapshot to a Release stream.

A Release stream can be used to expose a release to the next set of stakeholders, maybe customers, beta groups or in the situation of recursive systems of systems the next level up of development and integration. The Release stream can be used as the public face of the team’s output.

Integration across multiple SCM Streams

It’s hopefully obvious that you don’t want to slow down release points too much, ideally making a release once per sprint/iteration.

Some text from another blog of mine:

Teams are sometimes split into smaller sub-teams. Almost always they are part of a bigger team, which in turn is part of a bigger team and so on and so forth until we reach the organisational entity. Don’t get me started on B2B teams.  Imagine an onion, with lots of layers. A weird mutated onion with multiple cores and overlapping onions. One that has tendrils sticking out into other similarly mutated onions like a big brain… Ok metaphor taken too far. Back to the topic…

The point is that everything team is connected to everything else. At every boundary it’s necessary to have a “Definition of Done”…

More on Definition of Done here

Whatever the level of done is, the difference between that and the constant churn of development is useful to separate and streams can help you do that.

The practical bit

RTC Flow Diagram

To implement this sort of thing in Jazz SCM I’d recommend using RTC’s “post build deliver” functionality. The Post-Build deliver step (introduces in RTC 3.0.1.2) for Build definitions allows you to deliver the build snapshot to a stream. It works in a logical but sometimes counter-intuitive way…

Post-Build deliver does not deliver the results of the build process back to a stream (or another stream). That would cause a circular process of change-build-change-build etc. Also, the output of a build process is a set of derived artifacts, it’s unusual to keep these in an SCM system. Instead a build, or more likely a release, should be deployed either to a staging server or a binary repository (such as Nexus).

When a build takes place RTC makes a snapshot of the stream, accepts that to a build workspace then does the build. The post-build deliver step allows you to send the snapshot used to make the build to another stream.

Since this step can be controlled by build properties that means it can be turned on for a “Release” build programmatically  and left off generally.

RTC Post deliver options

Of course using this mechanism it’s possible to chain builds by having an integration build set up on the Integration Stream, which in turn could selectively auto-promote to another level if appropriate.

The only tricky thing about doing all of this is permissions. RTC still doesn’t have fine grained enough permissions for controlling this sort of thing, especially across project, it’s getting better though.

See Part 2 for Parallel branched development

See Part 3 for Component reuse

Top tips for RTC Jazz SCM:

  • Keep change sets small, deliver often
  • Baseline all the time, because you’ve made 3 changes, because you sneezed, because the minutes are a multiple of 10 or any other reason. Baselines are cheap and useful and it’s easier to revert to (or compare) a baseline than unpick a stack of changes.
  • Continuously build, unit test and monitor coverage, 1 or 2 out of three isn’t really Continuous Integration

If you want some expert SCM support, backup or help in your organisation get in touch.

Advertisements

6 responses to “Advanced RTC SCM Streaming – Part 1 Development, Integration and Release Streams

  1. Pingback: Advanced RTC SCM Streaming – Part 2 Parallel branched development

  2. Pingback: Advanced RTC SCM Streaming – Part 3 Component reuse « The Mac Daddy

  3. Pingback: Rational Team Concert source control | formtakt

  4. shruti November 25, 2013 at 10:42 am

    Hi,

    This article is really helpful, but in my organization instead of having different stream of integration they are having different project area for Integration. In this case how can I deliver change sets to Integration stream of different project area? I am using RTC 4.0 on RHEL 5.

    Thanks
    Shruti

    • mikemacd November 25, 2013 at 10:59 am

      Hi Shruti, you do it exactly the same way. You can have a workspace with flow targets (or streams with flow targets) that are to other streams outside of the project area, it doesn’t make any difference.

  5. edneubecker November 29, 2014 at 3:32 am

    Reblogged this on edneubecker and commented:
    A great blog about RTC SCM

What do you think?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: