Integrating Mercury Testing Tools with IBM Rational ClearQuest defects (and/or IBM Rational RequisitePro requirements)

Many organizations use Mercury “testing tools” for Quality Assurance.  It is also common for organizations to use IBM Rational ClearQuest for defect tracking, and IBM Rational RequisitePro for requirements management.  As development processes mature, organizations eventually seek a way to achieve traceability between testing and the rest of their processes and artifacts (defects, enhancement requests, requirements, source code, and design artifacts).  This discussion will examine two options available for establishing and maintaining traceability between test implementation tools and IBM Rational ClearQuest and RequisitePro via a test management tool.

To understand the available options, it is necessary to first categorize Mercury’s “testing tools” by function: test management (TestDirector or QualityCenter), versus test implementation (QuickTest Professional, WinRunner, LoadRunner). The IBM Rational equivalents for test managemet are "standalone" Test Manager, or ClearQuest Test Manager (CQTM) which is part of Rational ClearQuest 7.0. For test implementation, the IBM Rational equivalents are: Rational Functional Tester, Rational Manual Tester, Rational Performance Tester, Rational Tester for SOA Quality, and (legacy) Rational Robot.

There are two options for bridging Mercury test implementation tools with the rest of the development process, and the choice between the two hinges on whether Mercury is used for test management. However, for organizations which utilize a mixed-vendor test implementation toolset and require official, validated and supported integrations between tools, there may be only one feasible choice.

Option 1: If an organization must use QualityCenter or TestDirector for test management, the only option is to use Mercury-provided synchronizers between test management and external tools.

Other tools (non-testing)	Integration	Test Management	Integration	Test Implementation
ClearQuest	Mercury, Synchronizer	QualityCenter or TestDirector	Mercury, out-of-the-box	QuickTest Pro, WinRunner, LoadRunner
RequisitePro	Mercury, Synchronizer

Pro’s	Con’s
Mature test management software	Synchronization takes time, involves duplication of data; works for some companies, doesn’t work for others; breaks customized ClearQuest form behaviors; breaks UCM (Unified Change Management)
Free, tight integration between test management and test implementation	Initial purchase and subsequent maintenance and support of test management tool is expensive
	Non-existent or unsupported integrations with IBM Rational test implementation tools.

Option 2:  The organization uses IBM Rational ClearQuest Test Manager (CQTM) for test management.

Other tools (non-testing)	Integration	Test Management	Integration	Test Implementation
ClearQuest	None necessary	ClearQuest TestManager(CQTM)	Ring-Zero Test Adapters	QuickTest Pro, WinRunner, LoadRunner
RequisitePro	Rational, out-of-the-box

Pro’s	Con’s
Free, tight integration between test management and RequisitePro requirements.  Free, ultra-tight integration between test management and ClearQuest defects, enhancements.  No synchronization is necessary.	Less-mature test management software
Test management software is free, since it is built into ClearQuest.	Expense of third-party (Ring-Zero Software) test adapters.
Free integration between test management and IBM Rational test implementation tools.

Some organizations which have already made investments in Mercury test management software may consider migrating to ClearQuest Test Manager (CQTM).  In general, migration consists of exporting the hierarchy of test assets (test plan, test cases, test scripts) from the Mercury database and importing them into CQTM.  The level of effort necessary for migration may differ, depending on which particular Mercury test management tool features being used.  For example, if TestDirector is being used for requirements management, additional software (RequisitePro) and migration effort may be required.  However, the benefit of the additional need outweighs the costs: RequisitePro is useful for requirements at all stages in the development process, while TestDirector’s requirements management capability is confined to testing.

Learn more about the tools mentioned in this article:

IBM Rational Functional Tester	http://www-306.ibm.com/software/awdtools/tester/functional/index.html
IBM Rational Manual Tester	http://www-306.ibm.com/software/awdtools/tester/manual/index.html
IBM Rational Performance Tester	http://www-306.ibm.com/software/awdtools/tester/performance/index.html
IBM Rational Tester for SOA	http://www-306.ibm.com/software/awdtools/tester/soa/index.html
IBM Rational Robot	http://www-306.ibm.com/software/awdtools/tester/robot/index.html
IBM Rational ClearQuest TestManager	http://www.ibm.com/developerworks/rational/library/07/0306_see_regazzoni/
HP Mercury QualityCenter	http://h71028.www7.hp.com/enterprise/cache/454070-0-0-225-121.html
HP Mercury WinRunner	http://www.mercury.com/us/products/quality-center/functional-testing/winrunner/
HP Mercury QuickTest Professional	http://www.mercury.com/us/products/quality-center/functional-testing/quicktest-professional/
Ring-Zero Test Adapters	http://www.ring-zero.com/cqtm-qtp http://www.ring-zero.com/cqtm-wr

Leveraging DOORS Requirements in a Distributed Environment

These days, organizations are increasingly mandated to have auditable, automated traceability which links requirements to development, change management, and testing.

However, globally distributed environments ("Global Development and Delivery", or GDD) are another prevalent trend. In a not-so-hypothetical situation, requirements are managed by a team in the United States. Development and testing may be performed elsewhere, sometimes in another city within the United States, but more often in an entirely different country.

If we examine the tools being used for the software development lifecycle, it's usually Telelogic DOORS or IBM Rational RequisitePro for requirements. Often IBM Rational ClearQuest is being used for change management. Mercury QuickTest Pro, Loadrunner, and Winrunner own the testing space.

For a distributed team to leverage requirements throughout the lifecycle, it needs to find a way to (a) allow distributed workers access to requirements, (b) allow teams to respond to changes in requirements, and (c) establish automated traces between requirements to other development artifacts: tests, defects, enhancement requests, etc.

Telelogic DOORS is hands-down the tool of choice for requirements management. It's a great tool for the task, but unfortunately it performs very poorly over the WAN. It doesn't have a great web interface. It doesn't have any sort of multi-site replication capabilities. In terms of training users, it can be a bit cryptic and hard to learn for new users or users who are generally used to working with Word documents for requirements. Lastly, it's extremely expensive -- prohibitively expensive for companies who have built globally distributed teams to save money on labor. These companies would save money on labor, but ultimately spend the difference and more on licensing.

Is Citrix the anwer? It seems to work fairly well for distributed teams who are just using Citrix for DOORS. However, Citrix costs money too. More importantly, tools which are running on a local desktop cannot integrate with tools being used through a remote desktop window.

Enter IBM Rational ClearQuest. It is fairly ubiquitous in organizations. It is extremely flexible and customizable. It has security contexts and such for controlling access to records. It has a decent web interface. As of Version 7.0, IBM has added free, built-in test management ("ClearQuest Test Management", or CQTM) capability. And... it has multi-site replication capabilities.

Ring-Zero Software ClearTrace for DOORS is absolutely the best way to tie everything together. One one level, it's a scaleable, bidirectional integration between IBM Rational ClearQuest and Telelogic DOORS. However, its value really becomes apparent when it is applied to the Global Development and Delivery paradigm. It leverages ClearQuest as a vehicle for distributing DOORS requirements to distributed teams, while also maintaining automated traceability between requirements and other artifacts. Users are actually interacting through ClearQuest, and interface they likely already know how to use. It doesn't require testers to install DOORS on their desktops or learn how to use DOORS. However, if they have DOORS installed locally, it leverages it as well. When ClearTrace for DOORS is combined with other Ring-Zero products, such as Test Adapter for Mercury QuickTest Pro or Test Adapter for WinRunner, it shines even more. These additional integrations allow users of Mercury's testing tools leverage DOORS requirements as well, and from the same familiar ClearQuest interface.

To learn more about these solutions, please visit Ring-Zero Software.

Integrating ClearQuest and DOORS

Aimed at IT shops using Telelogic DOORS for requirements management and IBM Rational ClearQuest for issue tracking, Ring-Zero Software has released an immensely useful integration between the two tools: ClearTrace for DOORS. Before ClearTrace, the integrations available were very weak: some were monstrosities which barely qualified as "integrations" (one can only assume they existed for sales opportunities which were contingent on satisfying a "checkbox"), while others were ultimately deemed unusable by many larger organizations.

Ring-Zero leveraged its many years of building bridges between DOORS and Rational Software tools, and has provided its customers with a high-quality integration which the industry was desperately in need of.

From the Ring-Zero web site:

ClearTrace is described as a system which integrates ClearQuest and DOORS together, rather than as a simple mechanism for creating links between the two, for two reasons: First, ClearTrace provides simple, native access to data from both DOORS and ClearQuest. When local clients for both are present, users can access cross-environment functionality. Second, ClearTrace has been designed to provide scaleable access to “real world” data volumes which may span multiple databases.

Visit Ring-Zero Software's website and read more about ClearTrace for DOORS:
http://www.ring-zero.com/cq-doors

TestDirector or ClearQuest Test Manager (CQTM)?

There's no doubt about it: Mercury Interactive (now part of HP) tools such as WinRunner, QuickTest Professional, and LoadRunner are some of the most popular tools for automated software testing. However, for the remainder of the development lifecycle, many organizations have adopted IBM Rational tools such as ClearCase and ClearQuest for software configuration management, RequisitePro for requirements, Rational Software Architect (RSA) for modeling, and the list goes on. In short, companies today are typically using Mercury tools for testing and everything else from IBM Rational.

Testing and the other components of the development process are bridged by a test management tool, and again both vendors have product offerings which fill this need: Mercury makes two popular tools for test management, TestDirector (legacy product) and QualityCenter, while IBM Rational makes Test Manager (legacy product) and it's newly-released ClearQuest Test Manager (CQTM), which is a test management tool based on ClearQuest, its widely-adopted, highly-customizable, and enterprise-validated tool for change management.

As organizations seek for the "holy grail" of software development, an end-to-end integrated toolset comprised of the tools their users' prefer to use and the ones which work (or work best) in their development and deployment environments, they are often presented with a tough decision -- bridging testing with the rest of the development process means choosing between the two vendors' test management tools.

Each tool carries its own list of pro's and con's. However, software integration vendor Ring-Zero Software has simplified this choice by giving software development organizations more options and flexibility in making this decision. Two of Ring-Zero's most recent integrations are Test Adapter for WinRunner and Test Adapter for QuickTest Pro, add-ons for IBM Rational ClearQuest Test Manager (CQTM), Rational Application Developer, and Rational Functional Tester which bridges IBM Rational software tools and processes with Mercury Interactive WinRunner. These integrations are particularly attractive to organizations who would like to avoid the cost and complexities of using Mercury TestDirector and QualityCenter, with its synchronization-based integrations with IBM Rational tools, whilt at the same time leveraging the popular ClearQuest tool for Test Management. The integrations are also interesting to testing departments which use mixed vendor toolsets due to the simple fact that some tools work better than others under certain environments or against particular applications under test. Also, for testers transitioning from Mercury to Rational functional testing tools, these integrations can be lifesavers. Testers are often encouraged to create new tests using Rational tools, but testers still need to maintain the ability to leverage past work and maintain the ability to execute Mercury tests.

Perhaps most important, these integrations are commercial and supported by a company with business relationships with most of the primary tool vendors and a track record of supporting its products across hundreds of global Fortune-100/500 companies. Mercury's test management tools do not have official support for driving IBM Rational testing tools. What's more, Mercury often claims to offer integrations which allow execution of tests via IBM Rational tools, yet a simple Google search paints a different picture. One might conclude that these integrations either (a) don't exist, or (b) are "dirty little secrets" which are not officially maintaned or supported. How many organizations are willing to bet the enterprise on vaporware or hacks?

ClearQuest Test Manager is our preferred choice for test management for many reasons. IBM Rational has kept its interfaces open and enlisted the help of experienced ISV's to build commercial, supported integrations with competitors' tools. CIO's and CTO's are happy - organizations are able to further leverage the familiar change management tool which has already been widely validated in the enterprise. CFO's are happy because choosing ClearQuest means saving potentially hundreds of thousands, or millions, of dollars: CQTM is available for free with ClearQuest 7.0. Testers are happy because they are free, thanks to the Mercury integrations developed by Ring-Zero Software, to use Rational, Mercury, or both vendors' testing tools. CQTM's open interfaces guarantees that organizations are not left stranded without options if they decide to augment or replace their testing tools in the future.

Functional Testing Tools - Mixed Vendor Implementations

The plethora of functional testing tools previously mentioned (Mercury Interactive WinRunner, QuickTest Professional, Segue SilkTest, Compuware TestPartner, IBM Rational Robot, IBM Rational XDE/Functional Tester, and AccordSQA SmarteScript), combined with the fact that some tools work better than others in certain environments or against particular target applications, often creates a need for users to switch tools or at a minimum bring in a new tool to use in parallel with their current tool.

Perhaps a company used one vendor's functional testing tool to test Version 1.0 of their target application, and that tool worked fine with the technologies in 1.0, but now Version 2.0 uses different technologies and that testing tool doesn't support some particular aspect of 2.0. Maybe their previous vendor of choice hasn't released an updated version that works with today's technologies, yet works fine for continuing to support 1.0 revisions.

Perhaps an organization is outsourcing its testing and finding that one of the ramifications of outsourcing is a need to employ a new tool that allows greater reuse of testing assets. That might mean moving from a programmer-centric script-based testing tool to a simpler, keyword-driven testing tool. This may be further driven by a need to shorten the learning curve so that many new testers can learn a particular tool, ramp up quickly, and shorten the gap between hiring on and becoming a productive QA team member.

Change is sometimes necessary. What isn't necessary is completely abandoning an old tool, especially one that works fine for previous (but still maintained and supported) releases and one which has been invested in significantly in terms of man-hours spent creating test assets that work.

Ring-Zero Software sells Test Execution Adapters for IBM Rational TestManager. These adapters allow functional testing tools from a variety of tool vendors to be tied to Rational TestManager. Scripts can be executed and results captured within TestManager, and test results can be tied to requirements and defects. In the big picture, these adapters allow non-Rational functional testing tools to be integrated into the environments of companies that use predominantly Rational tools.

IBM Rational includes integrations out of the box that allow TestManager to work with:

• IBM Rational Robot
• IBM Rational Functional Tester
• IBM Rational Performance Tester
• IBM Rational Visual Test
• Command-Line driven tests
• Java-based tests
• VB-based tests

At the time of this post, Ring-Zero supports the following tools:

• Mercury Interactive WinRunner
• Mercury Interactive QuickTest Professional
• Segue SilkTest
• Compuware TestPartner
• AccordSQA SmarteScript

Most importantly, the adapters allow testing tools to coexist within a QA environment. In the event that two particular tools cannot coexist on the same machine, the IBM Rational Test Agent can be used to execute scripts on a remote machine. This means that QA shops can now retain Tool X as they fully or partially implement Tool Y, and that neither tool ends up being orphaned from the rest of the software development process. Of course, the catch is that these adapters require the use of IBM Rational TestManager for test planning. This catch really isn't so bad, however, because the tool also provides a bridge to requirements and defects.

More information on Test Script Execution Adapters is available from Ring-Zero Software.

Functional Testing Tools - traceability to requirements

There are quite a few functional testing tools out there: Mercury WinRunner, QuickTest Professional, SilkTest, TestPartner, IBM Rational Robot, IBM Rational XDE/Functional Tester, AccordSqa SmarteScript, etc. The list doesn't really go on and on; a few of the tools listed (Mercury and IBM Rational tools) own the lion's share of the functional testing tools market.

These functional testing tools do a pretty admirable job of enabling software testers to automate functional testing. Depending on the environment and target application being tested, some tools work better than others. When used in an isolated sense, however, the tools are missing a big piece of the picture: testing what needs to be tested in terms of requirements specifications. Without linking tests back to requirements, generating reports that show what's being tested (or not) and communicate becomes an error-prone manual process. Without an automated link back to requirements, it's hard to know how test results relate to the overall quality of software. In a nutshell, traceability from test cases and test results back to requirements provides context necessary for the QA process to work.

Ring-Zero Software makes integrations that provide this context. Because the company's founders came from IBM / Rational Software, they chose to build integrations that center around Rational's test planning tool, TestManager. Out of the box, Rational provides integrations that let testers tie their tests back to requirements kept in Rational's own requirements tool, IBM Rational RequisitePro, as well as IBM Rational Rose and Microsoft Excel. TestManager has an open architecture and published API that makes it possible to extend Test Manager's ability to tie to other requirements tools and functional testing tools.

Ring-Zero has added to Rational's list of integrations by building bridges to several popular requirements tools made by Rational competitors:

Ring-Zero Test Input Adapter for Telelogic DOORS Allows Rational TestManager users to work against the market-leader in requirements management tools, DOORS. Ring-Zero's integration leverages DOORS requirements by supporting "suspicion", which basically means that if you link test cases to requirements and the requirements change, the test cases are marked as "suspect" indicating that they may need review since the underlying requirements may have changed in ways that could affect testing. Ring-Zero also leverages DOORS requirements and letting them be used to populate Rational's ManualTest test scripts.

Ring-Zero Test Input Adapter for Borland CaliberRM TestManager users can work against Borland's entry into requirements management tools. This tool used to be made by StarBase before Borland bought them. This integration is very similar to their DOORS integration, but with CaliberRM instead of DOORS.

Ring-Zero Test Input Adapter for IBM Rational ClearQuest This is an interesting integration. Usually one would think of submitting defects from test results as a one-way process. Rational has a pretty tight integration with their testing software that lets you do this, but Ring-Zero has gone a step further. Often testers need a way to tie tests back to defects that have been submitted, perhaps to generate reports that demonstrate that all defects for a particular release are being tested. Ring-Zero's integration lets test planners treat defects like requirements in this sense, and if tests linked to defects fail, they can submit more defects.

More information about Ring-Zero Test Input Adapters for TestManager is available from Ring-Zero Software.