Feedback Rational edge e-zine July 2003: ClearCase performance

[Simon Yuen]

see http://www.therationaledge.com/content/jul_03/t_ccperformance_tm.jsp

I found this article presents a very good details of ClearCase RPC
performance issues.

However, they mention a ClearCase customer have replaced a complex
home grown tracking system with UCM, so that the overall performance
is improved.
I found that customer shifts the system performance issues to the
logical integration issues. Let do a search of "UCM +complain" from
google.

I am very disagree the sentense at the end of the article: "UCM is IBM
Rational's "best practices" process for managing change ...".

Rational (or Atria) suggests (since ClearCase ver 2) their customer to
create a branch per bug. First, this shows off ClearCase unlimited
branching and parallel branching power. Second, this recommandation
carry over to become UCM.

After 12 years as development process controller, and 7 years
ClearCase expert, I found the development process is like this:
1) At the beginning, using /main/LATEST and no need to control.

2) After the first formal system integration and testing, we can state
in one branch. /main/LATEST is the most common one.

3) When more system test, more test cases, more regression test, we
should enable the submission control by bug number into one branch.
- Since all the developers using one branch to continue the
development, the source code continues to integrate as soon as the
file be checked out and edit.
- I call this "Early Integration" method or Rational calls this
sequencal developement.
- We then use the list of bug numbers to generate the build config
spec.

4) In the pre-release or post-release state, no new code is allowed.
All the feature and bug codes should be done in a short live parallel
branch. May be this is the time to use UCM or parallel branch.
- I call this "Late Integration" method.
- In the Late Integration, the integration problem all happen in the
build time (merge time or delivery time).
- Once all the codes are merged into the integration branch, it is
very difficult to exclude (take out) one bug.

Therefore, UCM is not the "best practice" process. UCM 'may be' is
one of the choice since it using the Late Integration development
model.

One interesting thing is that IBM internally, in some development
groups, still using their home grown CMVC change management system.
They be using CMVC for more than 15 years and produce all the
wonderful product (including the whole AIX OS). However, CMVC is
using the Early Integration model.

Now, do you think UCM is the "best practices" ?

Simon Yuen

[Frank Schophuizen]

> Now, do you think UCM is the "best practices" ?


The term "best practice" is always arguable.

Concerning the "late integration" or "early integration" appraoch, I think
that UCM does support early integration:
- a developer joins a project
- he solves a bug
- he syncs with the integration and solves integration problems
- he delivers it
- he solves another bug
- he syncs with the integration and solves integration problems
- he delivers it
- only after the integrator has verified and accepted the changes, they are
made available to the rest of the team

Of course, you can also do a late integration with UCM:
- a developer joins a project
- he solves a lot of bugs and implements a lot new features
- he syncs with the integration and solves integration problems (which may
be huge)
- he delivers it
- only after the integrator has verified and accepted the changes, they are
made available to the rest of the team (which may be a hell of a job because
a lot has changed)

And you can do something in between by solving a number of issues before
delivering. And IF a single developer needs to work on different issues
(e.g. on bug fixes and on new features), he can join multiple times and use
multiple views (and hopefully he will not make a change in the wrong view
accidentally).

And when multiple developers have to work on the same issue, they can create
sub-stream of a development stream, work on different views on the same
stream or work on the same view altogether. The latter two are "continuous
integration" (since a change is integrated and made available when it is
checked in) and the former is "controlled integration" (either early or
late).


You are right that it is not "a branch per bug" approach. But I don't really
care as long as the development speed and the quality are high.

BTW, working with more than 100 people on a single branch (/main/LATEST) has
proven to be counterproductive and extremely difficult to control.


Frank.

[Simon Yuen]

"Frank Schophuizen" <fs5628@hotmailnospam.com> wrote in message news:<OlMRa.458714$V9.35601584@amsnews02.chello.com>... Now, do you think UCM is the "best practices" ?

I just think UCM is one of the choice but not the best.
The term "best practice" is always arguable. Concerning the "late integration" or "early integration" appraoch, I think that UCM does support early integration: - a developer joins a project - he solves a bug - he syncs with the integration and solves integration problems - he delivers it - he solves another bug - he syncs with the integration and solves integration problems - he delivers it - only after the integrator has verified and accepted the changes, they are made available to the rest of the team

In the above simple one developer case, I think this demo UCM as a
perfect integration process.
Of course, you can also do a late integration with UCM: - a developer joins a project - he solves a lot of bugs and implements a lot new features - he syncs with the integration and solves integration problems (which may be huge) - he delivers it - only after the integrator has verified and accepted the changes, they are made available to the rest of the team (which may be a hell of a job because a lot has changed) And you can do something in between by solving a number of issues before delivering. And IF a single developer needs to work on different issues (e.g. on bug fixes and on new features), he can join multiple times and use multiple views (and hopefully he will not make a change in the wrong view accidentally). And when multiple developers have to work on the same issue, they can create sub-stream of a development stream, work on different views on the same stream or work on the same view altogether. The latter two are "continuous integration" (since a change is integrated and made available when it is checked in) and the former is "controlled integration" (either early or late).

For my previous client (7x4), we try all the above UCM working models.
However, none of them work out OK. We even try more then that.

Their general problem:
When the upper layer developers waiting for a good baseline which
should delivery by the lower layer developers. However, the lower
layer developers are fighting with each other (resolve the header
files conflict) in the late integration model. After we spend few
days to resolve all the conflicts in the lower layer, the upper layer
developers are happy to rebase and continue their development.

After some tests, we need to take out some codes from the lower layer.
(Remember, we need to integrate to merge and spend time to test the
code.) However, in the late integration model, it is very diffcult to
take out or to undo some merged code (from development stream to
integration stream.) In alternative, it is very time consuming to
undo all the merged code and perform the integration again. We
already spend few days to get a good baseline.

The upper layer developers are very un-happy with the situation since
they cannot continue their works (the baseline is no longer here.)
The managment is very un-happy with all the delay. Basically, all
the invoked already do their best. Then, what is the problem ?

In the above situation, neither early nor late integration can help.
The problem for my client is: It is not the time to use late
integration (branch per bug). As I mention in my previous posting,
they are back to simple early integration (one branch for all). After
they spend few months to stablize their code and they are ready to
move on to UCM (no new major code, each problem should be isolated,
...., etc.)
You are right that it is not "a branch per bug" approach. But I don't really care as long as the development speed and the quality are high. BTW, working with more than 100 people on a single branch (/main/LATEST) has proven to be counter productive and extremely difficult to control.

As I mention above, IBM's CMVC uses single integrate branch approach
(early integration). And,
IBM using CMVC in AIX OS development which is 1000+ developers and
more than 5 sites.
IBM using CMVC in XLC C++ compiler development which is 500+
developers and more than 3 sites.
IBM using CMVC in WebSphere server development which is more then few
hundred developers and more than 3 site.
I can continue this list for few pages. Do you still think IBM
Rational UCM is the "best practice" ? Or UCM is one of the choice.

If you can spend some time to study CMVC (single branch) control model
(track), you may re-think the above statement.

I am not try to insult anybody, however I like to quote a good say:
A fool with a tool still a fool.

We should use our tool wisely. CMVC is not good in parallel
development. Currently, I still think ClearCase is the number one CM
tool which is so powerful, so flexible and expensive. I wish the
Subversion can support dynamic workspace (dynamic view) to compete
with ClearCase.
Frank.

[Frank Schophuizen]

> I can continue this list for few pages. Do you still think IBM Rational UCM is the "best practice" ? Or UCM is one of the choice.

If you asked my opinion, you get my opinion.
Rational does not claim that UCM is a "best practice", because it is not. It
is merely Rational's solution for best practices. In other words, it is
based on best practices and it gives support to best practices. But may be,
it is not the only solution for best practices.

I am not try to insult anybody, however I like to quote a good say: A fool with a tool still a fool.

"Beauty is in the eyes of the beholder"
translation: a best practice is not a universal, context-free concept

Currently, I still think ClearCase is the number one CM tool which is so powerful, so flexible and expensive.

The most expensive part of ClearCase is its powerfulness and flexibility.
Many companies have spent many manyears to develop a CM model and are still
spending many manyears to maintain it now. And many companies still have a
sub-optimal CM process that is very rigid, but they are so dependent on it
that they don't dare to take the risk of changing it for the better. That's
what makes it expensive and that's what I call "bad practice"!

With UCM is a predefined and scalable model, with an implementation on
Clearcase. If you don't like the UCM model, don't use it and don't try to
transform it.


Frank.

[NahtMy ReelName]

Simon Yuen wrote:
[...] We should use our tool wisely. CMVC is not good in parallel development. Currently, I still think ClearCase is the number one CM tool which is so powerful, so flexible and expensive. I wish the Subversion can support dynamic workspace (dynamic view) to compete with ClearCase.

And so it can, if you want to invest time or money in it.

It isn't like virtual file systems are rocket science. If you want to
apply the long view and appreciate some of the tenets of open source,
take those clearcase dollars and invest it in virtual file system
interfaces to subversion. Better, invest in a standards driven API to
the VFS capabilities and then multiple version management products can
use it.

Of course very few companies take that kind of long term open-source
favorable view, they just keep forking millions of dollars to
Rational-now-IBM.

I'm not an open source developer and I don't play one on TV. I'm just
pointing out that if you have wishes for subversion, the could no doubt
be expedited with time and money from you or your company.

[Dwain Wilder]

NahtMy ReelName wrote: Simon Yuen wrote: [...] We should use our tool wisely. CMVC is not good in parallel development. Currently, I still think ClearCase is the number one CM tool which is so powerful, so flexible and expensive. I wish the Subversion can support dynamic workspace (dynamic view) to compete with ClearCase. And so it can, if you want to invest time or money in it. It isn't like virtual file systems are rocket science. If you want to apply the long view and appreciate some of the tenets of open source, take those clearcase dollars and invest it in virtual file system interfaces to subversion. Better, invest in a standards driven API to the VFS capabilities and then multiple version management products can use it. Of course very few companies take that kind of long term open-source favorable view, they just keep forking millions of dollars to Rational-now-IBM.

This seems to be a Syrene song that Simon would best strap himself to themast
against, as Odysseus did millennia ago. It is quite a temptation to thinkthat
the best solution when you need a special tool need is to get to work androll
your own. But you should beware that this is to embark on a business thatother
businesses have made their primary objective. Do you really have the staff,
resources, and management committment to see such a venture through? To most
enterprises, it would be an expensive diversion, given the on-going costsover
the tool's life-cycle.

Dwain
I'm not an open source developer and I don't play one on TV. I'm just pointing out that if you have wishes for subversion, the could no doubt be expedited with time and money from you or your company.


--
Dwain Wilder
Bear Meadow Folk Instruments
http://www.bearmeadow.com
_______________________________
I arise in the morning torn between a desire to improve (or save) the world and
a desire to enjoy (or savor) the world. This makes it hard to plan the day.
—E. B. White

[NahtMy ReelName]

Dwain Wilder wrote: NahtMy ReelName wrote: Simon Yuen wrote: [...] We should use our tool wisely. CMVC is not good in parallel development. Currently, I still think ClearCase is the number one CM tool which is so powerful, so flexible and expensive. I wish the Subversion can support dynamic workspace (dynamic view) to compete with ClearCase. And so it can, if you want to invest time or money in it. It isn't like virtual file systems are rocket science. If you want to apply the long view and appreciate some of the tenets of open source, take those clearcase dollars and invest it in virtual file system interfaces to subversion. Better, invest in a standards driven API to the VFS capabilities and then multiple version management products can use it. Of course very few companies take that kind of long term open-source favorable view, they just keep forking millions of dollars to Rational-now-IBM. This seems to be a Syrene song that Simon would best strap himself to the mast against, as Odysseus did millennia ago. It is quite a temptation to think that the best solution when you need a special tool need is to get to work and roll your own. But you should beware that this is to embark on a business that other businesses have made their primary objective. Do you really have the staff, resources, and management committment to see such a venture through? To most enterprises, it would be an expensive diversion, given the on-going costs over the tool's life-cycle. Dwain

Hopefully readers realize my suggestion was somewhat tongue-in-cheek.

On the other hand, many Rational products are
effectively put into feature stasis after acquisition by Rational.
There have been numerous years where Clearcase changed very little from
a feature standpoint. (Just ask any customer who used it from 1994-1999).

Does this mean that Clearcase met user's needs? Hardly.

My point then is that there are still many problems remaining to be
solved in the version and change management field. Innovation from
established players is minimal, they just rake in the profits and put
very little into useful R&D on existing products.

If the established products do NOT meet your needs, and your
needs are sufficiently large in scope and costly in their existence,
then funding a small company or open source effort to solve your
problems isn't necessarily a bad idea.

It really depends on the size of your wallet and the cost to your
business in not solving a problem that is present but not addressed by
the 800lb gorillas of the market. Bear in mind that large enterprises
may pay upward of $10 million a year in clearcase licenses and
maintenance on an annual basis. That's a lot of moolah if things like
errors from config specs or branch-crazed development are delaying your
product cycles or increasing the odds of error-prone code bases.

Obviously some people are just more comfortable with setting the
expectation bar so low that anybody can trip over it (I'm not referring
to people in this discussion, just a generality). I'm of the belief
that we should strive for better solutions rather than ship crappy and
often delayed software because we are stuck in the trap of using the
industry equivalent of stone knives and bear skins.

Subversion probably isn't going to radically alter the way software is
done. But even if all it ever is is "a better cvs", well, that's
progress, and this newsgroup is documented proof that we need some
progress with respect to cvs :-) There would be even more progress with
the funding of a few more developers on it.

(I have no affiliation with Subversion. I'm just a rebel for the CM
cause and have never found clearcase to be a particularly satisfying
solution. I've also seen spectacular misuses of it, the corporate waste
boggles the mind.)

[Simon Yuen]

Be honest, the topic is out of what I want to say.

I am a ClearCase consultant for 8 years, my client actually pay me to
solve their ClearCase problem: build, branching, control, admin and
UCM. I think ClearCase is still the #1 CM tool and very good
product. Just my opinion, different situation should using different
solution. May be Frank is right (in term of using English wording):
"it is based on best practices and it gives support to best practices.
But may be, it is not the only solution for best practices." I want
to stop my say here.

In Subversion case, today, it is a better CVS tool with directory
versioning. Subversion may require a lot of work before it can
compete with (or even close to) ClearCase. However, it is exactly the
same in many year ago for Linux.

In my opinion, the most difficult task for CM field is not the
technical side. It is the management support. Most management want
the result ASAP. Therefore, if we implement UCM in the wrong place
and in the wrong time, it exactly slow down the end result. So, it
ends up losing management support. In some cases, the sequential
development is not too bad at the beginning, and then we can move on
to UCM when the code is stable and need isolation development.

That is it. I am out of this topic. Thanks and Bye.

Simon

[Stephen Baynes.]

"Simon Yuen" <simonyuen1999@hotmail.com> wrote in message
news:dbe4f7da.0307171845.42127534@posting.google.com... see http://www.therationaledge.com/content/jul_03/t_ccperformance_tm.jsp Rational (or Atria) suggests (since ClearCase ver 2) their customer to create a branch per bug. First, this shows off ClearCase unlimited branching and parallel branching power. Second, this recommandation carry over to become UCM.

It could also be read as showing off the limitations of Clearcase branching
and parallel development.
Why branch when you might find out later that you did not actually need it?
Mainly because you can't branch only when you need it and then decide later
which branch is the main line and which is merged in. So you create a branch
in case and always merge back.

Branching at a file level should be a detail handled by the tool that you
should not need to spend a lot of time planning how to manage it. Clearcase
has a lot of good features - but this is not one of them, it forces you to
spend a lot of effort on managing the triva of branches most of which are
not needed - at least UCM handles this for you, though it has to jump
through hoops to do it.

--
Stephen Baynes CEng MBCS

[Koushik Banerjee]

"NahtMy ReelName" <ergo@notavalidaddress.com> wrote in message
news:xKeSa.84740$OZ2.15438@rwcrnsc54...
: Simon Yuen wrote:
: [...]
: > We should use our tool wisely. CMVC is not good in parallel
: > development. Currently, I still think ClearCase is the number one CM
: > tool which is so powerful, so flexible and expensive. I wish the
: > Subversion can support dynamic workspace (dynamic view) to compete
: > with ClearCase.
:
: And so it can, if you want to invest time or money in it.
:
: It isn't like virtual file systems are rocket science. If you want to
: apply the long view and appreciate some of the tenets of open source,
: take those clearcase dollars and invest it in virtual file system
: interfaces to subversion. Better, invest in a standards driven API to
: the VFS capabilities and then multiple version management products can
: use it.
:
: Of course very few companies take that kind of long term open-source
: favorable view, they just keep forking millions of dollars to
: Rational-now-IBM.
:
: I'm not an open source developer and I don't play one on TV. I'm just
: pointing out that if you have wishes for subversion, the could no doubt
: be expedited with time and money from you or your company.
:
:
:
:
We use Clearcase but only the base clearcase, No UCM. We have
something developed in-house which acts similar to UCM, but the "best
practices" were defined by us through the tool. We have till a month
back using a customization where we always released from the main
branch, and then create branches for the product version which are
released. But now we use another customization(this also has GUI in Tk)
which the managers realize would help the developer in doing things
faster. This customization works on the belief that no element should be
touched on the main branch. Each product version would be a branch, and
each product version is a parent to one and child to another product.
So all branches sprout from one branch, and then another branch sprout
from it. This also helps us in particular with a concept with child
changeset. That is once you close a changeset in one branch, the
changeset is then made available in the child environment/branch. It is
then optional to actually close or delete it, but the artifacts to get the
changes from a parent environment to its child is available. I am not
aware whether it is possible with UCM to do the same, because it might
be possible that Rational did not hear or think of this to be good
one(unless it happens to exist, my experience with UCM is nill).

So it is true, each tool is meant to satisfy or provide a certain range
of solutions. We cannot ask more out of it, leaving aside the cost
factor.

Koushik