A SUPPLEMENT TO OCTOBER 1, 2006 Leading Tools And Solutions For Software Quality Assurance
A SUPPLEMENT TO
OCTOBER 1 2006
Leading ToolsAnd SolutionsFor SoftwareQualityAssurance
Testing is essential in everysoftware development
project Yet while computerscience programs teach devel-opers about software architec-ture object-oriented designalgorithms and programmingthey offer little formal empha-sis on testing and qualityassurance
Every day managers im-plore their programmers tocode faster But do they pro-vide them with the tools andknowledge they need to writebetter code Do they equiptheir test teams with the bestresources to stamp out defectsand vulnerabilities All toooften testing is neglected atevery stage
Testing is an art A develop-er or tester needs to understandwhere quality assurance fitsinto the application develop-ment life cycle and why therersquosmore to quality than passing atest suite or meeting some arbi-trary metric
Testing is a science Overthe past 30 years a significant
body of knowledge has evolvedbest practices for functionaltesting unit testing and per-formance testing but nowthose classic techniques arebeing augmented by recentinnovations in the field ofdefect management testautomation software config-uration management metricstest design and securityvul-nerability testing
Great software requiresgreat tools and great serviceproviders ldquoThe Art amp Scienceof Software Testingrdquo profilesleading testQA solutions tohelp you choose the right part-ners for your projects Thesecompanies can help yourdevelopment and test teamsmake better software
We hope you enjoy thisspecial testQA supplement toSD Times ampamp
1 october 2006 the artampamp science of software testing u 3
The ArtampampScienceOf Software Testing
Editorial DirectorAlan Zeichickalanbzmediacom
Managing EditorPatricia Saricapsaricabzmediacom
Art DirectorLuAnn T Palazzo
Copy EditorLaurie OrsquoConnell
Lead WriterGeorge Walshgwalshbzmediacom
Customer ServiceSD Times Subscriptions+1-847-763-9692sdtimeshalldatacom
Article ReprintsLisa Abelson +1-516-379-7097labelsonbzmediacom
BZ Media LLC7 High Street Suite 407Huntington NY 11743+1-631-421-4158 bull fax +1-631-421-4130wwwbzmediacom bull infobzmediacom
PresidentTed Bahr
Executive Vice PresidentAlan Zeichick
Cover Photograph by Elena Korenbaum
A SUPPLEMENT TO SD TIMESOCTOBER 1 2006
Copyright copy 2006 BZ Media LLCAll Rights Reserved
SUBSCRIBE TODAYwwwsdtimescom
ArtScienceBoth
Platinum
Thanks to Our Sponsors
Diamond
Gold
Alan Zeichick
Editorial Director SD Times
4 CEO PERSPECTIVERethinking Our Approach to Software QualityADAM KOLAWA CO-FOUNDER AND CEO PARASOFT
6 DIAMOND SPONSORParasoft Catches Bugs Before They HatchmdashWith AutomationPARASOFT
11 TEST SOLUTIONS LEADERIBM
15 APPLICATION LIFE CYCLE MANAGEMENTTechExcel Takes the Guesswork Out of Quality AssuranceTECHEXCEL
17 DEFECT TRACKINGAxosoft Keeps Development Projects on the Fast TrackAXOSOFT
19 SOFTWARE CONFIGURATION MANAGEMENTPerforce Makes SCM Fast Easy and First-ClassPERFORCE
21 WEB SECURITYKeeping the Bad Guys at Bay With Cenzic SolutionsCENZIC
23 TESTING SERVICESStelligent Brings Objectivity to Quality MeasurementSTELLIGENT
introduction
t a b l e o f c o n t e n t s
The primary mission of information technology is toincrease profits through improved business processesCompanies are constantly rethinking and struggling
with how to use IT to a competitive advantage reduce IT oper-ating and maintenance costs and reduce the total cost of own-ershiphellipall while attempting to deliver increased value
Most of these challenges are directly linked to makingsoftware workmdashwithout incurring unreasonable costs Manypeople in the industry would agree that low IT productivi-ty is the reason software development is so costly But whyare IT teams with all their expertise and hard work suf-fering from low productivity The root cause of low produc-tivity is errors made throughout the software developmentlife cycle
These errors include everything from performance errorsto security errors to misimplemented functionality to errorsthat crash an entire system They essentially stifle IT teamsrsquoability to produce working software in a reasonable time andat reasonable costs In fact if you look at virtually any ITteam you will see that its members spend about 80 percentof their time chasing and fixing bugs and only about 20percent of their time on tasks that deliver value and improvethe business This practice is far from productive
Adding to this inefficiency is the traditional softwaredevelopment approach of leaving testing till late in the devel-opment life cycle It is only then that QA does the testingnecessary to ensure that bugs are found requirements metand reliability performance and security goals achievedbefore an application is deployed into production Findingand fixing errors late in the development cycle is exponen-tially more costly time-consuming and inefficient thanaddressing them early and throughout the cycle Maintainingthis approach is a primary reason that we continue to strug-gle with quality and low productivity in the software world
Many other industries have struggled with low qualityhigh costs and low productivity as a result of human errorThe automotive industry for example recognized that althoughmistakes cannot be entirely eliminated they can be controlledThose automotive manufacturers who by taking a holisticand preventative approach to the problem making quality anintegrated focus throughout the production process and evenmodifying their production lines to prevent as many errorsas possible from ever entering the products addressed theirmost critical problems and have remained viable
The software industry still has not learned this impor-
tant lesson Many people think that error prevention is notpossible in the software industry they believe that becauseeach piece of software is different the lessons learned fromworking on one piece of software cannot be applied to oth-er pieces Instead of trying to prevent errors from enteringsoftware the industry tries to test errors out at the end ofthe development life cycle
First we build a product then we test at the end of theproduction cycle to determine whether the product worksand finally we remove any errors that testing exposesThroughout this process we cross our fingers and hope thatthe most insidious and embarrassing problems will be iden-tified before the release However a consideration of thestaggering number and impact of software errors reportedannually and their cost to the US economy suggests thatthis quality-through-end-of-cycle-testing approach is notyielding the desired results
The belief that our traditional software testing approachcan create quality software is a fundamental problem Wedonrsquot think of the whole process of building and deployingsoftware in a way that would prevent errors because we donrsquotbelieve that it can actually be done Yet this error preven-tion approach is not only possible it is necessary The increas-
ing complexity of software systems the push for faster nearcontinuous release cycles and the expanding dependencyon software for nearly every phase of business executionrequire that error prevention be addressed
If the software industry is serious about reducing theerror rate and resolving the issues that stem from errors wecanrsquot afford to continue hoping that our current approachto testing will miraculously start yielding quality softwareInstead we need to follow in the footsteps of other indus-
4 t the artampamp science of software testing 1 october 2006
Rethinking Our ApproachTo Software Quality
Testing in general must
become the responsibility
of every team member
c e o p e r s p e c t i v e
1 october 2006 the artampamp science of software testing u 5
tries and start preventing errors throughout the softwaredevelopment life cycle
Achieving Software QualityAchieving a consistently high level of quality starts with com-mitting to a practice of an end-to-end quality process Whilethere is no single silver bullet for producing reliable high-quality software there are proven steps that software devel-opment organizations can and should employ to help preventsoftware errors and improve development productivity Themost significant of these are addressed below
Establish a Quality Initiative And Group Culture Organizations need to establish a group culture that places ashared focus and importance on quality Many companies con-tinue to treat development and testing as independent disci-plines This separation of development and QA leads to manysoftware problems and inefficienciesmdashdevelopers might writecode assuming that someone in the QA department will injectquality into the software When problems occur the code justbounces back and forth between departments without anyonetaking responsibility
The manufacturing world learned long ago that it couldnot separate the responsibility of production from theresponsibility of verification and expect to achieve qualityThese responsibilities go hand in hand Organizations shouldplace development and QA under the same managementor merge the two groups completely to facilitate owner-
ship and responsibility for code quality In an establishedgroup culture developers will show that they care about
the code because caring about the code is synonymouswith caring about the group
Adopt Quality PracticesOrganizations must adopt software error preven-
tion practices from the earliest stages of devel-opment Well-known software error prevention
practices such as coding standards unit test-ing and regression testing are not regularly
followed despite their recognized value in catch-ing errors at the code level early in the cycle when it
is easiest and least costly to find and fix them These prac-tices should be an integrated part of an organizationrsquos devel-opment process Testing in general must become the respon-sibility of every team member
The trend toward service enablement of applications(eg Web services SOA) and faster shorter software releasecycles mandates that quality assurance and testing can nolonger be treated as a set event handled as an independ-ent discipline and relegated to a single proscribed phaseof the development life cycle It must become a continu-ous integrated part of the development process enabledthrough the application of known software development
quality practices
AutomateEffective adoption of software quality practicesrequires automation of as many testing tasks as possibleDevelopment organizations are increasingly burdened toproduce more code faster and with the same or fewerresources If they are to adopt software quality practicesthey will need to use software testing tools and developmentmethodologies that allow them to automate many of thesepractices and integrate them easily into their developmentprocesses Fortunately there are a growing number of prod-uct offerings that deliver this automation and integrationand that allow developers to take a ldquotest-as-you-coderdquoapproach to developmentmdashto readily create reusable testobjects and operate within a framework that facilitates cre-ating high-quality software
If their companies are to remain competitive softwareorganizations must improve their productivity by control-ling their costs while ensuring the high quality of their deliv-erables To do so requires rethinking and restructuring theway we have traditionally approached software developmentand delivery and necessitates a heightened commitment toquality throughout the software development life cycle and
Adam KolawaCo-Founder and CEO Parasoft
Everyone knows that bugs exist But at what stage of thedevelopment cycle should they be isolated and killed
Parasoft says ldquoTest early and often when bugs are easiestand least costly to find and fixrdquo Parasoftrsquos Jtest software forJava developers uses automation to make testing fast easy
and practical for developersto perform during the de-velopment life cyclersquos cod-ing stages when testing is usually the furthestthing from a program-merrsquos mind
ldquoIf you look at adeveloperrsquos resume thelast thing yoursquoll see list-ed is testing becausedevelopers just donrsquotlike to testrdquo says BrianHunt Parasoftrsquos VP of sales and actingCOO (wwwparasoftcom) ldquoHowever atsome point youhave to prove that
the software worksWe help developersvalidate what theyrsquove
built from the point of cre-ation to the completion of development That
validation starts at the desktop in the same way that spellcheckers are run against text documentsrdquo
Jtest provides Java development teams an automated unittesting and code analysis tool suite that performs compre-hensive test and analysis of Java source code exposing bugsand errors in code structure execution and design at thesource or unit level Used as a plug-in to the developerrsquos IDE(such as Eclipse WSAD Rational RAD) or integrated witha central build process Parasoft Jtest is designed to be usedby development teams in a ldquotest-as-you-coderdquo strategy tofind and eliminate errors early in the development processbefore they can infect the main application codebaseldquoUncovering errors early and at their source or root causemakes them quicker and less costly to fix and helps revealand resolve design errors that could have extended negativeimpact on an application if gone undetectedrdquo states Hunt
The newly released Jtest 80 adds new testing innovationsto help teams automatically verify the functionality of com-
plex constantly changing enterprise systems like Java EESOA and Web services reducing the risks of system down-time and security vulnerabilities At the same time teamscan find more defects with their existing resources increas-ing productivity while adhering to budget parameters
One of the most exciting new features of Jtest is itsBugDetective By automatically tracing and simulating exe-cution paths BugDetective exposes runtime defects thatwould be difficult or even impossible to find through man-ual testing or inspections With BugDetective you can nowfind diagnose and fix classes of software errors that routine-ly evade standard analysis and unit testing techniques
Jtest also lets development teams automatically generateand run tests using the popular Apache Cactus test frame-work This gives organizations early development-level defectexposure that might go unnoticed until QA deployment orproduction time when itrsquos a lot more expensive and prohib-itive to find and fix problems Another new technology isJtest Tracer which creates realistic functional JUnit test cas-es that reflect an applicationrsquos correct functional behaviorWith Jtest Tracer organizations can quickly create librariesof regression test cases that can be run to ensure that newcode changes donrsquot inadvertently break existing applicationfunctionality
ldquoThe key to reducing testing time is automationrdquo Huntdeclares ldquoJtest can even perform testing overnight to scanthe code find errors and report those errors to the devel-opers when they start working in the morning It lets themdrill straight through the results to the lines of code thatneed to be fixed It can also perform automatic functionaltests that run the code to make sure that it does what itrsquossupposed to do Because we write these tests in an openformat you can modify and extend them to meet your spe-
cific needsrdquoJtest integrates with com-
plementary Parasoft prod-ucts to provide automatedsystemwide testing solutions
for Web applications Web services and other n-tier systemsMoreover Jtest works as part of a comprehensive teamwideAutomated Error Prevention solution that provides central-ized administration and application of test practices man-agement dashboards and metrics for real-time analysis thathelp managers evaluate code compliance code readiness andteam productivity
With Jtest you catch bugs before they hatchmdashearly andoften ampamp
Parasoft Catches Bugs BeforeThey HatchmdashWith Automation
6 t the artampamp science of software testing 1 october 2006
d i a m o n d s p o n s o r
Brian Hunt
VP of Sales and Acting COO
SPONSORED WHITE PAPER
SPONSORED WHITE PAPER
SPONSORED WHITE PAPER
SPONSORED WHITE PAPER
When it comes to software testing two key words are speedand accuracymdashnot only of the application yoursquore test-
ing but also of the QA process itself Thatrsquos what teams needmdashand what TechExcel delivers in DevTest one of the three toolsin its DevSuite DevTest offers your developers test standardi-zation test reuse and powerful defect analysis capabilities whilegiving managers a birdrsquos-eye view of the entire process
ldquoDevTest is an integrated solution that allows QA teams toimprove standardization and leverage existing testing knowl-edge to carefully monitor test executionrdquo says Tieren Zhou TechExcelrsquos CEO and chief software architect ldquoDevTest accomplish-es that goal by focusing knowledge management test librarycreation planning and scheduling and test execution and analy-sis I think our approach is truly unique to the marketrdquo
DevTest is a one-stop information resource for your teamstoring all related test documentation including requirementsdocuments specifications automation scripts screen shotsand other essential components in a central repository This
ldquoknowledge viewrdquo can beused in test creation andexecution so that the testteam is always equipped to plan and execute itsassignments
A test library is built onthis knowledge founda-tion letting your teamreuse tests for new testassignments new soft-ware versions or evenentirely different soft-ware products Thesestandard test pro-cedures called test templates can belinked to supporting
documentation in theknowledge view Testtemplates can be or-
ganized and classifiedbased on products applicable
environments functional areas or any otherstructure on which a team needs to focus
Once a test library has been created QA managers and teamleads use a wizard-driven interface to assign tasks to test teamshelping them leverage everything in the test library includingprevious test assignments and defect history to aid in the plan-
ning process The test team receives its test assignment in theDevTest interface executes the items assigned to it and sub-mits defects directly from the interface into an integrated defect-tracking tool Meanwhile DevTest tracks the test results in areal-time dashboard and in presentation-quality custom reports
In addition to DevTest TechExcel (wwwtechexcelcom) offerstwo other products in the DevSuite DevTrack and DevPlan
DevTrack tracks andmanages product defectschange requests and oth-er issues facilitating team-work among users teams
and customers DevTrack also provides workflow and processautomation robust searching and reporting and point-and-click customization
The newest member of the DevSuite DevPlan is an inno-vative project-tracking tool designed exclusively for applicationlife-cycle management DevPlan unites project tracking andissue management and incorporates configurable workflowsnotifications meeting requests and process automation
Zhou explains that in order to be effective a QA organiza-tion needs both preparation and education and that TechExcelrsquostools can help
ldquoDevTest provides all of the materials needed to create andexecute test assignments in one location so that leads and man-agers have the information they need to craft effective testsand testers have the information they need to execute themrdquohe says ldquoDevTestrsquos planning wizards are a favorite amongtest managers because they allow them to query existing datato help plan their assignments and reduce guesswork In addi-tion the built-in integration with DevTrack allows testers toexecute their test assignments and log and regress defects froma single interfacerdquo
In the near future the DevSuite will support distributeddeployment allowing global development organizations toachieve the benefits of local performance even when workingon a global projectmdashstay tuned for more about this excitingdevelopment
DevTest can help both small and large organizations get ahandle on quality assurance ldquoDevTest is useful in any develop-ment environmentrdquo Zhou says ldquoItrsquos well suited for large glob-al development organizations because it provides them with ascalable real-time view of their test projects regardless of whetherthe tests are executed by their core team by an outsourced teamor by auto-testing tools However even smaller developmentteams benefit from the ability to create manage analyze andreuse their test coveragerdquo ampamp
TechExcel Takes the Guesswork Out of Quality Assurance
a p p l i c a t i o n l i f e c y c l e m a n a g e m e n t
gold sponsor
Tieren Zhou
CEO and Chief Software Architect
1 october 2006 the artampamp science of software testing u 15
The Fast amp Scalable Team Solution forDefect amp Issue Tracking bull Feature amp Change Tracking bull Task amp To-do List Tracking bull Helpdesk Ticket Tracking
OnTime is the market-leading project defect and feature management tool for agile software development and test teamsOnTime facilitates tracking analyzing and trending team-based software development efforts in an intuitive and powerful user interface A fully customizable UI powerful workflow process enforcements two-way email communications and custom reports combine to help software development teams ship software on-time
Available for Windows Web amp VSNET 20032005
800middot653middot0024
wwwaxosoftcomso f tware fo r so f tware deve lopment trade
Ship Software OnTimetrade
Only $495 for up to 5 Users bull Only $995 for up to 10 UsersFree Single-User Installations
OnTime 2006 Small Team Edition
bull For Teams up to 10 Membersbull Free Single-User Installationsbull $495 for 5-Team Membersbull $995 for 10-Team Members
OnTime 2006 Professional Edition
bull For Teams up to 10 Membersbull Free Single-User Installationsbull $495 for 5-Team Membersbull $995 for 10-Team Members
bull For Teams of 10 to 1000 Membersbull From $149 Per User
SDTimesAd_OnTime2006indd 1 62706 14151 PM
Managing the entire scope of one or more software devel-opment projects can be daunting Within the sea of
tasks features defects and test cycles among team mem-bers managers stakeholders and customers just figuringout whorsquos working on what can be a challenge Thatrsquos where
Axosoftrsquos OnTime 2006enters the picture
OnTime 2006 is adefect management sys-tem designed to help soft-ware development teamsship software on timeWith a focus on ship-ping software on timeit not only effectivelytracks and managesbugs it effectivelyaddresses the broad-er challenges andbest practices ofproject manage-ment
OnTime 2006offers maximum
flexibility for adminis-trators and users alikewith ready access from
a Windows client a Webbrowser or within Visual Studio The highly
configurable and customizable application can also be inte-grated with leading SCM packages including Perforce SCMSourceGear Vault Subversion and Visual SourceSafe
In addition to tracking defects features and tasks thesoftware governs projects with highly customizable workflowand security rules It also offers e-mail notifications and con-versation thread tracking time tracking and work loggingcustom fields and reporting audit trails and archiving
While many of Axosoftrsquos clients switch to OnTime 2006from other products utilizing its importing functionalityothers seek to upgrade from a manual process
ldquoTraditionally development tasks requests defects andother items that occur over the course of the developmentcycle have been tracked in spreadsheets providing ampleopportunity for human error and miscommunicationbetween teamsrdquo says Dan Suceava chief software architectfor Axosoft (wwwaxosoftcom)
ldquoOnTime 2006 keeps track of everything and prevents
important items from slipping through the cracksrdquo heexplains ldquoInstead of wasting developer time with trivialupdates and report requests managers can easily pull theinformation themselves OnTime 2006 frees developers frommost if not all of the process-related overhead that sur-rounds a project and it helps them focus on what they dobest building great softwarerdquo
Developers who use OnTime 2006 see a clear intuitiveview of all of their projects including all the issues defectsfeature requests milestones and tasks As they completetasks developers escalate those items to subsequent work-flow steps This can be set to automatically e-mail otherteam members when the next set of actions is ready to beperformed
OnTime 2006 allows project managers to define speci-fications workflows and security rules They can create ldquowhatifrdquo scenarios that generate predictions for completion timesworkload distribution and other milestones During the pro-jectrsquos execution project managers have access to total proj-ect visibility They know who is working on what the progressbeing made on tasks where bottlenecks are occurring defectrates and estimated completion times
After a project has been completed managers can takeadvantage of the accurate project history that continues toreside within OnTime 2006 This information can be usedfor anything from measuring various productivity rates toproviding a basis for decisions concerning future projectsThus past projects become part of a living knowledge basethat can be consulted at any time
Axosoft even offers a Customer Portal add-on for OnTime2006 that embraces customers as participants in the processof shipping software on time It provides a Web interfacetied to the OnTime database where customers can submit
bugs and other inputCustomizable security set-tings determine how muchproject information will bevisible to customers This
functionality is especially useful for consultants and ISVsduring beta-testing phases
More functionality is in the works for the productldquoOnTime 2006 is already a tool designed to enable the entiredevelopment team to ship software on timerdquo explainsSuceava ldquoWhile today it meets the needs of project man-agers developers and testers extremely well future versionswill provide further functionality for support professionalsIT directors and executivesrdquo ampamp
Axosoft Keeps DevelopmentProjects on the Fast Track
1 october 2006 the artampamp science of software testing u 17
d e f e c t t r a c k i n g
gold sponsor
Dan Suceava
Chief Software Architect
Software configuration management isnrsquot a luxurymdashforany modern software development team itrsquos a necessi-
ty But when an SCM system is as easy to use easy to admin-ister and full of productivity-enhancing features and bene-fits as Perforcersquos system itrsquoll feel like all of your developersare traveling in first class all the way
ldquoThe Perforce SCM System lets teams of local and dis-tributed developers share project files that are centrally storedand managed by the Perforce Serverrdquo says John WalkerPerforcersquos principal product consultant ldquoThe Perforce Serverhandles user requests and tracks all development activity inthe built-in Perforce database Each filersquos state informationcan be quickly discerned from any of Perforcersquos cross-platform clients This means that a user can see when a ver-sion of a particular file has been updated deleted or addedto the server Icons associated with the files indicate whethertheyrsquore currently being edited added or deleted by otherdevelopersrdquo
The Perforce SCM System(wwwperforcecom) letseach developer on your teamobtain and refresh a localprivate copy of versionedfiles by synchronizing themwith files stored in a filedepot Since all of themetadata is centralizedfile state information canbe gathered quickly andeasily Perforce efficient-ly manages both bina-ry and text files fromsource code to graph-ics to documentationThatrsquos one reasonwhy Perforce has
been adopted not onlyby pure developmentshops but also by
chip and hardwaremanufacturers that maintain
large binary assets For example Walker says the Perforce System is very pop-
ular in the game development industry where artists arerequired to create and store large numbers of large imagefiles To help all usersmdashnot just software developersmdashthePerforce software automatically renders thumbnail versions
of image files that can be viewed from within the applica-tionrsquos cross-platform graphical client P4V Regardless ofwhether yoursquore working with text or binary artifacts Perforcehandles the job with style
Perforcersquos clientserverapplication operates overany network or the In-ternet and includes itsown internal database
enginemdashsaving you money improving performance and get-ting the system up and running fast Installing the PerforceServer is simple Download the installation set from the com-panyrsquos Web site and run it Installing any one of a varietyof Perforce clients is also straightforward and the resourcesneeded to manage the installation are minimal Even largedistributed sites of 2000 or more users can be maintainedby a single administrator explains Walker
While Perforce has the basics down pat the real payoff isin the extras that the softwarersquos creators developed to makeyour own coders more efficient Perforcersquos intelligent branch-ing mechanism lets developers work on different releasebranches of a particular application in parallel Code linessupporting specific releases are clearly and visually definedin the system Once a branch is created the relationship itshares with the parent branch is tracked in the serverrsquos meta-data Since the ancestry of files is tracked between branch-es the integration history is maintained in the metadataThat makes it easy to see which changes have and havenrsquotbeen integrated between the related branchesmdashwhether yoursquoreworking in Java J2EE NET CC++ C Visual Basic orHTML
Perforce provides a basic defect-tracking system called jobsA job typically represents an enhancement request or a bugto be fixed Job definitions are customizable to support work-flow and jobs can also work with leading third-party defecttracking systems Support for Mercuryrsquos Quality Center isplanned for the next release This integration will allow usersto enter bugs in Quality Center and have them replicatedinto Perforce as jobs
ldquoThe Perforce SCM Systemrsquos high performance is the prod-uct of a streamlined architecture and closely integrated imple-mentationmdashnot expensive server or network hardwarerdquoWalker says ldquoThe Perforce Server does not require dedicat-ed hardware and client workstations never need upgradingThe systemrsquos networking capabilities arenrsquot a significant loadfor a typical LAN With little need for customization or con-figuration you can be up and running in minutesrdquo ampamp
Perforce Makes SCM FastEasy and First-Class
s o f t w a r e c o n f i g u r a t i o n m a n a g e m e n t
gold sponsor
John Walker
Principal Product Consultant
1 october 2006 the artampamp science of software testing u 19
The faster you find a security problem in the application lifecycle the better If developers catch vulnerabilities early they
can be fixed prior to pushing apps off to QA and customersFixing problems after they slip into a product can cost far morethan catching them early And of course if you donrsquot find the
defects someone else willldquoCenzic solutions provide
tremendous efficiencies andan immediate return on investmentrdquo says JohnWeinschenk Cenzicrsquospresident and CEO (wwwcenziccom) ldquoBy auto-mating the security test-ing process our cus-tomers can secure theirapplications faster andless expensively Likeperformance andfunctionality testingsecurity testingshould be automat-ed Vulnerabilitiesare an open invita-
tion to hackers AllWeb applications andinfrastructure can be
tested with Cenzic solutionsto locate security problemsrdquo
Cenzicrsquos two key security products are Cenzic Hailstorm andCenzic ClickToSecure These solutions provide the most com-prehensive and accurate tests in the industry Weinschenk saysthat while most vendors use scanning technology to focus onlyon commonly known vulnerabilities Hailstorm uses Cenzicrsquospatent-pending Stateful Assessment technology to automatepenetration testing by using a series of transactions to identifyvulnerabilities
Plus Hailstormrsquos SmartAttack Objects Library takes thesoftware beyond merely finding security holesmdashit can also helpenforce internal policies as well as bring organizations intoregulatory compliance with rules like the Gramm-Leach-BlileyAct (GLBA) SB1386 and Sarbanes-Oxley and the PaymentCard Industry (PCI) Data Security Standard and with bestpractices including SANS and OWASP in addition to manyother regulatory standards
ClickToSecure is a service that allows developers to make useof Software as a Service (SaaS) Tests are conducted remotely by
Cenzicrsquos own security expertsmdashand that means that your devel-opment team can build secure software even if it doesnrsquot haveexpertise in that area ClickToSecure accesses applicationsusing a combination of Hailstorm technology and the CenzicIntelligent Analysis (CIA) Research Lab to run assessments
In fact that CIA Lab is the foundation of the SmartAttackObjects Library Cenzic provides continuous updates based onnew vulnerabilities through the lab similar to an anti-virusmodel to help you stay ahead of the exploits and attacks Youcan use the library just as it is plus you can use CenzicrsquosSmartAttack Objects as templates that your own security archi-tects can customize for your organizationrsquos special require-ments if necessary
In addition to its software applications and services Cenzicoffers security training courses that give customers the knowl-edge and skills to use and maintain its products successfullyThe company also offers consulting services that include on-site assessment methodology and software-engineering andconsulting implementation with professional vulnerabilityconsultants who have experience in penetration testing andethical hacking
Among Cenzicrsquos many customers are Boston College DebtExchange IRIS Link and K2 Networks Boston Collegersquos devel-opment group uses Hailstorm in-house throughout its infor-mation security group to test all university Web applicationsAs they find security weaknesses developers use Hailstormreports to remediate those vulnerabilities The same group alsoaddresses its regulatory compliance issues with Hailstorm
In the case of Debt Exchange IRIS Link and K2 NetworksClickToSecure is put to work to perform security assessmentsCustomers call or fax requests to test their applications andCenzic experts test them remotely using Hailstorm in collabo-ration with the CIA Labrsquos expertise Detailed results are then
presented to the customeralong with detailed remedi-ation information
Weinschenk says thatCenzic solutions are a perfect fit for development managersand other executives Using a dashboard customers can viewapplications in the testing phase as well as the number andtypes of vulnerabilities that are found
Weinschenk adds that testing is extremely importantbecause susceptibility to intrusion can result in major recoverycosts and regulatory penalties He also reports that Cenzic hasnt tested a single application that was not vulnerable Butdonrsquot worrymdashCenzic finds the vulnerabilities before the badguys do Put Cenzic to work for you ampamp
Keeping the Bad Guys At Bay With Cenzic Solutions
1 october 2006 the artampamp science of software testing u 21
w e b s e c u r i t y
John Weinschenk
President and Chief Executive Officer
gold sponsor
Donrsquot MissOut
On Another Issue of The
Test amp QAReport
e-newsletter
Each FREE weekly issue includes original articles that interview top thought leaders in software testingand quality trends best practices
and testQA methodologiesGet must-read articles that appear
only in this e-newsletter
Sign up atwwwstpmagcomtqa
Many tools exist that help QA departments and programmerstest applications However itrsquos also crucial that senior-
level software managers have the necessary information to inter-act with executive management and accurately assess the effec-tiveness of their development teams ldquoIf addressing software qual-ity early in the development cycle is an important priority for yourcompany Stelligent can helprdquo says Burke Cox Stelligentrsquos CEO
ldquoAs the thought leaders in early testing we use commercialand open-source technologies to assess and manage softwarequality during the development and assembly of applicationsrdquoCox says ldquoThe Quality Risk Index provides an objective meas-ure of software quality that enables an entire organization toassess quality and progressrdquo
Cox believes that Stelligent (wwwstelligentcom) transformsthe way software is developed and tested By introducing com-prehensive inspection as part of a continuous process both thedevelopment team and senior management gain crucial visibil-ity into the quality risks associated with software projects This
real-time feedback lets organ-izations manage quality longbefore products are everdelivered to the QA team forevaluation
Stelligentrsquos services suchas its Kickstart Quality RiskAssessment are used byorganizations developingsoftware using managedlanguages like C orJava providing bothstatic and dynamicanalysis on the appli-cation source andcompiled code
How effective isthe Kickstart Quality
Risk Assessment Wellone customer recentlyacquired a software
product that it believedwas of high quality based on a
due-diligence assessment of the source code andend user experience However when trying to integrate the prod-uct into its larger solution it found the product had poor toler-ance to change
Thatrsquos when the customer called Stelligent whose KickstartQuality Risk Assessment identified the problems Then its
Quality Roadmap and Seven-Point Implementation Plan identified top priorities for making programmatic changes through refactoring and introduced developer testing as an active part of the continuous integration process The customer
found that Stelligentrsquos QualityRoadmap quickly solved theproblem and then its Con-tinuous Quality program kept
them solved Case closedldquoFor many organizations our Quality Risk Index is the first
objective measure of their software quality that they have everseenrdquo Cox says ldquoThis helps the software manager by providingan independent assessment of what their quality practices cur-rently achieve When working with the executive managementteam in developing schedules approving budgets and other prod-uct management activities the software manager uses Stelligentreports as a basis for discussion For example high code com-plexity with poor test coverage might make a compelling argu-ment to delay release as opposed to simply stating that the prod-uct is just not ready for prime timerdquo
Stelligentrsquos services provide a critical and objective meas-urement of your softwarersquos quality Use this intelligence tomake it difficult to enter new defects into the source repos-itory helping your developers leverage the continuous qual-ity feedback before code even reaches the QA department
Stelligent is a subsidiary of JNetDirect which makesConvergence a solution for providing real-time visibility intosoftware quality CoView which helps your team develop accu-rate and effective JUnit tests and also high-performance JDBCdrivers But long before JNetDirect acquired Stelligent it was acustomer using the Kickstart Quality Risk Assessment servicesto help build its own products Stelligentrsquos impact was so dra-matic that JNetDirect immediately saw the value of bringingStelligentrsquos services to a broader audience
ldquoDefects will always enter the build systemrdquo Cox says ldquoOurbusiness is ensuring that they cannot live there for very longQuality may not have a material impact on initial product rev-enue but it is the most significant driver of product profitabil-ity Most companies struggle with describing quality it is mixedwith subjective measures and anecdotal evidence Our customershave a quality score they can point to when setting goals andbenchmarks For organizations interested in increasing earningslowering the costs associated with poor product quality is thebest place to startrdquo
Stelligentrsquos services start with the Kickstart Quality RiskAssessmentmdashask the company how its objective measures canhelp you too improve your software quality ampamp
Stelligent Brings ObjectivityTo Quality Measurement
t e s t i n g s e r v i c e s
Burke Cox
Chief Executive Officer
1 october 2006 the artampamp science of software testing u 23
silver sponsor
Testing is essential in everysoftware development
project Yet while computerscience programs teach devel-opers about software architec-ture object-oriented designalgorithms and programmingthey offer little formal empha-sis on testing and qualityassurance
Every day managers im-plore their programmers tocode faster But do they pro-vide them with the tools andknowledge they need to writebetter code Do they equiptheir test teams with the bestresources to stamp out defectsand vulnerabilities All toooften testing is neglected atevery stage
Testing is an art A develop-er or tester needs to understandwhere quality assurance fitsinto the application develop-ment life cycle and why therersquosmore to quality than passing atest suite or meeting some arbi-trary metric
Testing is a science Overthe past 30 years a significant
body of knowledge has evolvedbest practices for functionaltesting unit testing and per-formance testing but nowthose classic techniques arebeing augmented by recentinnovations in the field ofdefect management testautomation software config-uration management metricstest design and securityvul-nerability testing
Great software requiresgreat tools and great serviceproviders ldquoThe Art amp Scienceof Software Testingrdquo profilesleading testQA solutions tohelp you choose the right part-ners for your projects Thesecompanies can help yourdevelopment and test teamsmake better software
We hope you enjoy thisspecial testQA supplement toSD Times ampamp
1 october 2006 the artampamp science of software testing u 3
The ArtampampScienceOf Software Testing
Editorial DirectorAlan Zeichickalanbzmediacom
Managing EditorPatricia Saricapsaricabzmediacom
Art DirectorLuAnn T Palazzo
Copy EditorLaurie OrsquoConnell
Lead WriterGeorge Walshgwalshbzmediacom
Customer ServiceSD Times Subscriptions+1-847-763-9692sdtimeshalldatacom
Article ReprintsLisa Abelson +1-516-379-7097labelsonbzmediacom
BZ Media LLC7 High Street Suite 407Huntington NY 11743+1-631-421-4158 bull fax +1-631-421-4130wwwbzmediacom bull infobzmediacom
PresidentTed Bahr
Executive Vice PresidentAlan Zeichick
Cover Photograph by Elena Korenbaum
A SUPPLEMENT TO SD TIMESOCTOBER 1 2006
Copyright copy 2006 BZ Media LLCAll Rights Reserved
SUBSCRIBE TODAYwwwsdtimescom
ArtScienceBoth
Platinum
Thanks to Our Sponsors
Diamond
Gold
Alan Zeichick
Editorial Director SD Times
4 CEO PERSPECTIVERethinking Our Approach to Software QualityADAM KOLAWA CO-FOUNDER AND CEO PARASOFT
6 DIAMOND SPONSORParasoft Catches Bugs Before They HatchmdashWith AutomationPARASOFT
11 TEST SOLUTIONS LEADERIBM
15 APPLICATION LIFE CYCLE MANAGEMENTTechExcel Takes the Guesswork Out of Quality AssuranceTECHEXCEL
17 DEFECT TRACKINGAxosoft Keeps Development Projects on the Fast TrackAXOSOFT
19 SOFTWARE CONFIGURATION MANAGEMENTPerforce Makes SCM Fast Easy and First-ClassPERFORCE
21 WEB SECURITYKeeping the Bad Guys at Bay With Cenzic SolutionsCENZIC
23 TESTING SERVICESStelligent Brings Objectivity to Quality MeasurementSTELLIGENT
introduction
t a b l e o f c o n t e n t s
The primary mission of information technology is toincrease profits through improved business processesCompanies are constantly rethinking and struggling
with how to use IT to a competitive advantage reduce IT oper-ating and maintenance costs and reduce the total cost of own-ershiphellipall while attempting to deliver increased value
Most of these challenges are directly linked to makingsoftware workmdashwithout incurring unreasonable costs Manypeople in the industry would agree that low IT productivi-ty is the reason software development is so costly But whyare IT teams with all their expertise and hard work suf-fering from low productivity The root cause of low produc-tivity is errors made throughout the software developmentlife cycle
These errors include everything from performance errorsto security errors to misimplemented functionality to errorsthat crash an entire system They essentially stifle IT teamsrsquoability to produce working software in a reasonable time andat reasonable costs In fact if you look at virtually any ITteam you will see that its members spend about 80 percentof their time chasing and fixing bugs and only about 20percent of their time on tasks that deliver value and improvethe business This practice is far from productive
Adding to this inefficiency is the traditional softwaredevelopment approach of leaving testing till late in the devel-opment life cycle It is only then that QA does the testingnecessary to ensure that bugs are found requirements metand reliability performance and security goals achievedbefore an application is deployed into production Findingand fixing errors late in the development cycle is exponen-tially more costly time-consuming and inefficient thanaddressing them early and throughout the cycle Maintainingthis approach is a primary reason that we continue to strug-gle with quality and low productivity in the software world
Many other industries have struggled with low qualityhigh costs and low productivity as a result of human errorThe automotive industry for example recognized that althoughmistakes cannot be entirely eliminated they can be controlledThose automotive manufacturers who by taking a holisticand preventative approach to the problem making quality anintegrated focus throughout the production process and evenmodifying their production lines to prevent as many errorsas possible from ever entering the products addressed theirmost critical problems and have remained viable
The software industry still has not learned this impor-
tant lesson Many people think that error prevention is notpossible in the software industry they believe that becauseeach piece of software is different the lessons learned fromworking on one piece of software cannot be applied to oth-er pieces Instead of trying to prevent errors from enteringsoftware the industry tries to test errors out at the end ofthe development life cycle
First we build a product then we test at the end of theproduction cycle to determine whether the product worksand finally we remove any errors that testing exposesThroughout this process we cross our fingers and hope thatthe most insidious and embarrassing problems will be iden-tified before the release However a consideration of thestaggering number and impact of software errors reportedannually and their cost to the US economy suggests thatthis quality-through-end-of-cycle-testing approach is notyielding the desired results
The belief that our traditional software testing approachcan create quality software is a fundamental problem Wedonrsquot think of the whole process of building and deployingsoftware in a way that would prevent errors because we donrsquotbelieve that it can actually be done Yet this error preven-tion approach is not only possible it is necessary The increas-
ing complexity of software systems the push for faster nearcontinuous release cycles and the expanding dependencyon software for nearly every phase of business executionrequire that error prevention be addressed
If the software industry is serious about reducing theerror rate and resolving the issues that stem from errors wecanrsquot afford to continue hoping that our current approachto testing will miraculously start yielding quality softwareInstead we need to follow in the footsteps of other indus-
4 t the artampamp science of software testing 1 october 2006
Rethinking Our ApproachTo Software Quality
Testing in general must
become the responsibility
of every team member
c e o p e r s p e c t i v e
1 october 2006 the artampamp science of software testing u 5
tries and start preventing errors throughout the softwaredevelopment life cycle
Achieving Software QualityAchieving a consistently high level of quality starts with com-mitting to a practice of an end-to-end quality process Whilethere is no single silver bullet for producing reliable high-quality software there are proven steps that software devel-opment organizations can and should employ to help preventsoftware errors and improve development productivity Themost significant of these are addressed below
Establish a Quality Initiative And Group Culture Organizations need to establish a group culture that places ashared focus and importance on quality Many companies con-tinue to treat development and testing as independent disci-plines This separation of development and QA leads to manysoftware problems and inefficienciesmdashdevelopers might writecode assuming that someone in the QA department will injectquality into the software When problems occur the code justbounces back and forth between departments without anyonetaking responsibility
The manufacturing world learned long ago that it couldnot separate the responsibility of production from theresponsibility of verification and expect to achieve qualityThese responsibilities go hand in hand Organizations shouldplace development and QA under the same managementor merge the two groups completely to facilitate owner-
ship and responsibility for code quality In an establishedgroup culture developers will show that they care about
the code because caring about the code is synonymouswith caring about the group
Adopt Quality PracticesOrganizations must adopt software error preven-
tion practices from the earliest stages of devel-opment Well-known software error prevention
practices such as coding standards unit test-ing and regression testing are not regularly
followed despite their recognized value in catch-ing errors at the code level early in the cycle when it
is easiest and least costly to find and fix them These prac-tices should be an integrated part of an organizationrsquos devel-opment process Testing in general must become the respon-sibility of every team member
The trend toward service enablement of applications(eg Web services SOA) and faster shorter software releasecycles mandates that quality assurance and testing can nolonger be treated as a set event handled as an independ-ent discipline and relegated to a single proscribed phaseof the development life cycle It must become a continu-ous integrated part of the development process enabledthrough the application of known software development
quality practices
AutomateEffective adoption of software quality practicesrequires automation of as many testing tasks as possibleDevelopment organizations are increasingly burdened toproduce more code faster and with the same or fewerresources If they are to adopt software quality practicesthey will need to use software testing tools and developmentmethodologies that allow them to automate many of thesepractices and integrate them easily into their developmentprocesses Fortunately there are a growing number of prod-uct offerings that deliver this automation and integrationand that allow developers to take a ldquotest-as-you-coderdquoapproach to developmentmdashto readily create reusable testobjects and operate within a framework that facilitates cre-ating high-quality software
If their companies are to remain competitive softwareorganizations must improve their productivity by control-ling their costs while ensuring the high quality of their deliv-erables To do so requires rethinking and restructuring theway we have traditionally approached software developmentand delivery and necessitates a heightened commitment toquality throughout the software development life cycle and
Adam KolawaCo-Founder and CEO Parasoft
Everyone knows that bugs exist But at what stage of thedevelopment cycle should they be isolated and killed
Parasoft says ldquoTest early and often when bugs are easiestand least costly to find and fixrdquo Parasoftrsquos Jtest software forJava developers uses automation to make testing fast easy
and practical for developersto perform during the de-velopment life cyclersquos cod-ing stages when testing is usually the furthestthing from a program-merrsquos mind
ldquoIf you look at adeveloperrsquos resume thelast thing yoursquoll see list-ed is testing becausedevelopers just donrsquotlike to testrdquo says BrianHunt Parasoftrsquos VP of sales and actingCOO (wwwparasoftcom) ldquoHowever atsome point youhave to prove that
the software worksWe help developersvalidate what theyrsquove
built from the point of cre-ation to the completion of development That
validation starts at the desktop in the same way that spellcheckers are run against text documentsrdquo
Jtest provides Java development teams an automated unittesting and code analysis tool suite that performs compre-hensive test and analysis of Java source code exposing bugsand errors in code structure execution and design at thesource or unit level Used as a plug-in to the developerrsquos IDE(such as Eclipse WSAD Rational RAD) or integrated witha central build process Parasoft Jtest is designed to be usedby development teams in a ldquotest-as-you-coderdquo strategy tofind and eliminate errors early in the development processbefore they can infect the main application codebaseldquoUncovering errors early and at their source or root causemakes them quicker and less costly to fix and helps revealand resolve design errors that could have extended negativeimpact on an application if gone undetectedrdquo states Hunt
The newly released Jtest 80 adds new testing innovationsto help teams automatically verify the functionality of com-
plex constantly changing enterprise systems like Java EESOA and Web services reducing the risks of system down-time and security vulnerabilities At the same time teamscan find more defects with their existing resources increas-ing productivity while adhering to budget parameters
One of the most exciting new features of Jtest is itsBugDetective By automatically tracing and simulating exe-cution paths BugDetective exposes runtime defects thatwould be difficult or even impossible to find through man-ual testing or inspections With BugDetective you can nowfind diagnose and fix classes of software errors that routine-ly evade standard analysis and unit testing techniques
Jtest also lets development teams automatically generateand run tests using the popular Apache Cactus test frame-work This gives organizations early development-level defectexposure that might go unnoticed until QA deployment orproduction time when itrsquos a lot more expensive and prohib-itive to find and fix problems Another new technology isJtest Tracer which creates realistic functional JUnit test cas-es that reflect an applicationrsquos correct functional behaviorWith Jtest Tracer organizations can quickly create librariesof regression test cases that can be run to ensure that newcode changes donrsquot inadvertently break existing applicationfunctionality
ldquoThe key to reducing testing time is automationrdquo Huntdeclares ldquoJtest can even perform testing overnight to scanthe code find errors and report those errors to the devel-opers when they start working in the morning It lets themdrill straight through the results to the lines of code thatneed to be fixed It can also perform automatic functionaltests that run the code to make sure that it does what itrsquossupposed to do Because we write these tests in an openformat you can modify and extend them to meet your spe-
cific needsrdquoJtest integrates with com-
plementary Parasoft prod-ucts to provide automatedsystemwide testing solutions
for Web applications Web services and other n-tier systemsMoreover Jtest works as part of a comprehensive teamwideAutomated Error Prevention solution that provides central-ized administration and application of test practices man-agement dashboards and metrics for real-time analysis thathelp managers evaluate code compliance code readiness andteam productivity
With Jtest you catch bugs before they hatchmdashearly andoften ampamp
Parasoft Catches Bugs BeforeThey HatchmdashWith Automation
6 t the artampamp science of software testing 1 october 2006
d i a m o n d s p o n s o r
Brian Hunt
VP of Sales and Acting COO
SPONSORED WHITE PAPER
SPONSORED WHITE PAPER
SPONSORED WHITE PAPER
SPONSORED WHITE PAPER
When it comes to software testing two key words are speedand accuracymdashnot only of the application yoursquore test-
ing but also of the QA process itself Thatrsquos what teams needmdashand what TechExcel delivers in DevTest one of the three toolsin its DevSuite DevTest offers your developers test standardi-zation test reuse and powerful defect analysis capabilities whilegiving managers a birdrsquos-eye view of the entire process
ldquoDevTest is an integrated solution that allows QA teams toimprove standardization and leverage existing testing knowl-edge to carefully monitor test executionrdquo says Tieren Zhou TechExcelrsquos CEO and chief software architect ldquoDevTest accomplish-es that goal by focusing knowledge management test librarycreation planning and scheduling and test execution and analy-sis I think our approach is truly unique to the marketrdquo
DevTest is a one-stop information resource for your teamstoring all related test documentation including requirementsdocuments specifications automation scripts screen shotsand other essential components in a central repository This
ldquoknowledge viewrdquo can beused in test creation andexecution so that the testteam is always equipped to plan and execute itsassignments
A test library is built onthis knowledge founda-tion letting your teamreuse tests for new testassignments new soft-ware versions or evenentirely different soft-ware products Thesestandard test pro-cedures called test templates can belinked to supporting
documentation in theknowledge view Testtemplates can be or-
ganized and classifiedbased on products applicable
environments functional areas or any otherstructure on which a team needs to focus
Once a test library has been created QA managers and teamleads use a wizard-driven interface to assign tasks to test teamshelping them leverage everything in the test library includingprevious test assignments and defect history to aid in the plan-
ning process The test team receives its test assignment in theDevTest interface executes the items assigned to it and sub-mits defects directly from the interface into an integrated defect-tracking tool Meanwhile DevTest tracks the test results in areal-time dashboard and in presentation-quality custom reports
In addition to DevTest TechExcel (wwwtechexcelcom) offerstwo other products in the DevSuite DevTrack and DevPlan
DevTrack tracks andmanages product defectschange requests and oth-er issues facilitating team-work among users teams
and customers DevTrack also provides workflow and processautomation robust searching and reporting and point-and-click customization
The newest member of the DevSuite DevPlan is an inno-vative project-tracking tool designed exclusively for applicationlife-cycle management DevPlan unites project tracking andissue management and incorporates configurable workflowsnotifications meeting requests and process automation
Zhou explains that in order to be effective a QA organiza-tion needs both preparation and education and that TechExcelrsquostools can help
ldquoDevTest provides all of the materials needed to create andexecute test assignments in one location so that leads and man-agers have the information they need to craft effective testsand testers have the information they need to execute themrdquohe says ldquoDevTestrsquos planning wizards are a favorite amongtest managers because they allow them to query existing datato help plan their assignments and reduce guesswork In addi-tion the built-in integration with DevTrack allows testers toexecute their test assignments and log and regress defects froma single interfacerdquo
In the near future the DevSuite will support distributeddeployment allowing global development organizations toachieve the benefits of local performance even when workingon a global projectmdashstay tuned for more about this excitingdevelopment
DevTest can help both small and large organizations get ahandle on quality assurance ldquoDevTest is useful in any develop-ment environmentrdquo Zhou says ldquoItrsquos well suited for large glob-al development organizations because it provides them with ascalable real-time view of their test projects regardless of whetherthe tests are executed by their core team by an outsourced teamor by auto-testing tools However even smaller developmentteams benefit from the ability to create manage analyze andreuse their test coveragerdquo ampamp
TechExcel Takes the Guesswork Out of Quality Assurance
a p p l i c a t i o n l i f e c y c l e m a n a g e m e n t
gold sponsor
Tieren Zhou
CEO and Chief Software Architect
1 october 2006 the artampamp science of software testing u 15
The Fast amp Scalable Team Solution forDefect amp Issue Tracking bull Feature amp Change Tracking bull Task amp To-do List Tracking bull Helpdesk Ticket Tracking
OnTime is the market-leading project defect and feature management tool for agile software development and test teamsOnTime facilitates tracking analyzing and trending team-based software development efforts in an intuitive and powerful user interface A fully customizable UI powerful workflow process enforcements two-way email communications and custom reports combine to help software development teams ship software on-time
Available for Windows Web amp VSNET 20032005
800middot653middot0024
wwwaxosoftcomso f tware fo r so f tware deve lopment trade
Ship Software OnTimetrade
Only $495 for up to 5 Users bull Only $995 for up to 10 UsersFree Single-User Installations
OnTime 2006 Small Team Edition
bull For Teams up to 10 Membersbull Free Single-User Installationsbull $495 for 5-Team Membersbull $995 for 10-Team Members
OnTime 2006 Professional Edition
bull For Teams up to 10 Membersbull Free Single-User Installationsbull $495 for 5-Team Membersbull $995 for 10-Team Members
bull For Teams of 10 to 1000 Membersbull From $149 Per User
SDTimesAd_OnTime2006indd 1 62706 14151 PM
Managing the entire scope of one or more software devel-opment projects can be daunting Within the sea of
tasks features defects and test cycles among team mem-bers managers stakeholders and customers just figuringout whorsquos working on what can be a challenge Thatrsquos where
Axosoftrsquos OnTime 2006enters the picture
OnTime 2006 is adefect management sys-tem designed to help soft-ware development teamsship software on timeWith a focus on ship-ping software on timeit not only effectivelytracks and managesbugs it effectivelyaddresses the broad-er challenges andbest practices ofproject manage-ment
OnTime 2006offers maximum
flexibility for adminis-trators and users alikewith ready access from
a Windows client a Webbrowser or within Visual Studio The highly
configurable and customizable application can also be inte-grated with leading SCM packages including Perforce SCMSourceGear Vault Subversion and Visual SourceSafe
In addition to tracking defects features and tasks thesoftware governs projects with highly customizable workflowand security rules It also offers e-mail notifications and con-versation thread tracking time tracking and work loggingcustom fields and reporting audit trails and archiving
While many of Axosoftrsquos clients switch to OnTime 2006from other products utilizing its importing functionalityothers seek to upgrade from a manual process
ldquoTraditionally development tasks requests defects andother items that occur over the course of the developmentcycle have been tracked in spreadsheets providing ampleopportunity for human error and miscommunicationbetween teamsrdquo says Dan Suceava chief software architectfor Axosoft (wwwaxosoftcom)
ldquoOnTime 2006 keeps track of everything and prevents
important items from slipping through the cracksrdquo heexplains ldquoInstead of wasting developer time with trivialupdates and report requests managers can easily pull theinformation themselves OnTime 2006 frees developers frommost if not all of the process-related overhead that sur-rounds a project and it helps them focus on what they dobest building great softwarerdquo
Developers who use OnTime 2006 see a clear intuitiveview of all of their projects including all the issues defectsfeature requests milestones and tasks As they completetasks developers escalate those items to subsequent work-flow steps This can be set to automatically e-mail otherteam members when the next set of actions is ready to beperformed
OnTime 2006 allows project managers to define speci-fications workflows and security rules They can create ldquowhatifrdquo scenarios that generate predictions for completion timesworkload distribution and other milestones During the pro-jectrsquos execution project managers have access to total proj-ect visibility They know who is working on what the progressbeing made on tasks where bottlenecks are occurring defectrates and estimated completion times
After a project has been completed managers can takeadvantage of the accurate project history that continues toreside within OnTime 2006 This information can be usedfor anything from measuring various productivity rates toproviding a basis for decisions concerning future projectsThus past projects become part of a living knowledge basethat can be consulted at any time
Axosoft even offers a Customer Portal add-on for OnTime2006 that embraces customers as participants in the processof shipping software on time It provides a Web interfacetied to the OnTime database where customers can submit
bugs and other inputCustomizable security set-tings determine how muchproject information will bevisible to customers This
functionality is especially useful for consultants and ISVsduring beta-testing phases
More functionality is in the works for the productldquoOnTime 2006 is already a tool designed to enable the entiredevelopment team to ship software on timerdquo explainsSuceava ldquoWhile today it meets the needs of project man-agers developers and testers extremely well future versionswill provide further functionality for support professionalsIT directors and executivesrdquo ampamp
Axosoft Keeps DevelopmentProjects on the Fast Track
1 october 2006 the artampamp science of software testing u 17
d e f e c t t r a c k i n g
gold sponsor
Dan Suceava
Chief Software Architect
Software configuration management isnrsquot a luxurymdashforany modern software development team itrsquos a necessi-
ty But when an SCM system is as easy to use easy to admin-ister and full of productivity-enhancing features and bene-fits as Perforcersquos system itrsquoll feel like all of your developersare traveling in first class all the way
ldquoThe Perforce SCM System lets teams of local and dis-tributed developers share project files that are centrally storedand managed by the Perforce Serverrdquo says John WalkerPerforcersquos principal product consultant ldquoThe Perforce Serverhandles user requests and tracks all development activity inthe built-in Perforce database Each filersquos state informationcan be quickly discerned from any of Perforcersquos cross-platform clients This means that a user can see when a ver-sion of a particular file has been updated deleted or addedto the server Icons associated with the files indicate whethertheyrsquore currently being edited added or deleted by otherdevelopersrdquo
The Perforce SCM System(wwwperforcecom) letseach developer on your teamobtain and refresh a localprivate copy of versionedfiles by synchronizing themwith files stored in a filedepot Since all of themetadata is centralizedfile state information canbe gathered quickly andeasily Perforce efficient-ly manages both bina-ry and text files fromsource code to graph-ics to documentationThatrsquos one reasonwhy Perforce has
been adopted not onlyby pure developmentshops but also by
chip and hardwaremanufacturers that maintain
large binary assets For example Walker says the Perforce System is very pop-
ular in the game development industry where artists arerequired to create and store large numbers of large imagefiles To help all usersmdashnot just software developersmdashthePerforce software automatically renders thumbnail versions
of image files that can be viewed from within the applica-tionrsquos cross-platform graphical client P4V Regardless ofwhether yoursquore working with text or binary artifacts Perforcehandles the job with style
Perforcersquos clientserverapplication operates overany network or the In-ternet and includes itsown internal database
enginemdashsaving you money improving performance and get-ting the system up and running fast Installing the PerforceServer is simple Download the installation set from the com-panyrsquos Web site and run it Installing any one of a varietyof Perforce clients is also straightforward and the resourcesneeded to manage the installation are minimal Even largedistributed sites of 2000 or more users can be maintainedby a single administrator explains Walker
While Perforce has the basics down pat the real payoff isin the extras that the softwarersquos creators developed to makeyour own coders more efficient Perforcersquos intelligent branch-ing mechanism lets developers work on different releasebranches of a particular application in parallel Code linessupporting specific releases are clearly and visually definedin the system Once a branch is created the relationship itshares with the parent branch is tracked in the serverrsquos meta-data Since the ancestry of files is tracked between branch-es the integration history is maintained in the metadataThat makes it easy to see which changes have and havenrsquotbeen integrated between the related branchesmdashwhether yoursquoreworking in Java J2EE NET CC++ C Visual Basic orHTML
Perforce provides a basic defect-tracking system called jobsA job typically represents an enhancement request or a bugto be fixed Job definitions are customizable to support work-flow and jobs can also work with leading third-party defecttracking systems Support for Mercuryrsquos Quality Center isplanned for the next release This integration will allow usersto enter bugs in Quality Center and have them replicatedinto Perforce as jobs
ldquoThe Perforce SCM Systemrsquos high performance is the prod-uct of a streamlined architecture and closely integrated imple-mentationmdashnot expensive server or network hardwarerdquoWalker says ldquoThe Perforce Server does not require dedicat-ed hardware and client workstations never need upgradingThe systemrsquos networking capabilities arenrsquot a significant loadfor a typical LAN With little need for customization or con-figuration you can be up and running in minutesrdquo ampamp
Perforce Makes SCM FastEasy and First-Class
s o f t w a r e c o n f i g u r a t i o n m a n a g e m e n t
gold sponsor
John Walker
Principal Product Consultant
1 october 2006 the artampamp science of software testing u 19
The faster you find a security problem in the application lifecycle the better If developers catch vulnerabilities early they
can be fixed prior to pushing apps off to QA and customersFixing problems after they slip into a product can cost far morethan catching them early And of course if you donrsquot find the
defects someone else willldquoCenzic solutions provide
tremendous efficiencies andan immediate return on investmentrdquo says JohnWeinschenk Cenzicrsquospresident and CEO (wwwcenziccom) ldquoBy auto-mating the security test-ing process our cus-tomers can secure theirapplications faster andless expensively Likeperformance andfunctionality testingsecurity testingshould be automat-ed Vulnerabilitiesare an open invita-
tion to hackers AllWeb applications andinfrastructure can be
tested with Cenzic solutionsto locate security problemsrdquo
Cenzicrsquos two key security products are Cenzic Hailstorm andCenzic ClickToSecure These solutions provide the most com-prehensive and accurate tests in the industry Weinschenk saysthat while most vendors use scanning technology to focus onlyon commonly known vulnerabilities Hailstorm uses Cenzicrsquospatent-pending Stateful Assessment technology to automatepenetration testing by using a series of transactions to identifyvulnerabilities
Plus Hailstormrsquos SmartAttack Objects Library takes thesoftware beyond merely finding security holesmdashit can also helpenforce internal policies as well as bring organizations intoregulatory compliance with rules like the Gramm-Leach-BlileyAct (GLBA) SB1386 and Sarbanes-Oxley and the PaymentCard Industry (PCI) Data Security Standard and with bestpractices including SANS and OWASP in addition to manyother regulatory standards
ClickToSecure is a service that allows developers to make useof Software as a Service (SaaS) Tests are conducted remotely by
Cenzicrsquos own security expertsmdashand that means that your devel-opment team can build secure software even if it doesnrsquot haveexpertise in that area ClickToSecure accesses applicationsusing a combination of Hailstorm technology and the CenzicIntelligent Analysis (CIA) Research Lab to run assessments
In fact that CIA Lab is the foundation of the SmartAttackObjects Library Cenzic provides continuous updates based onnew vulnerabilities through the lab similar to an anti-virusmodel to help you stay ahead of the exploits and attacks Youcan use the library just as it is plus you can use CenzicrsquosSmartAttack Objects as templates that your own security archi-tects can customize for your organizationrsquos special require-ments if necessary
In addition to its software applications and services Cenzicoffers security training courses that give customers the knowl-edge and skills to use and maintain its products successfullyThe company also offers consulting services that include on-site assessment methodology and software-engineering andconsulting implementation with professional vulnerabilityconsultants who have experience in penetration testing andethical hacking
Among Cenzicrsquos many customers are Boston College DebtExchange IRIS Link and K2 Networks Boston Collegersquos devel-opment group uses Hailstorm in-house throughout its infor-mation security group to test all university Web applicationsAs they find security weaknesses developers use Hailstormreports to remediate those vulnerabilities The same group alsoaddresses its regulatory compliance issues with Hailstorm
In the case of Debt Exchange IRIS Link and K2 NetworksClickToSecure is put to work to perform security assessmentsCustomers call or fax requests to test their applications andCenzic experts test them remotely using Hailstorm in collabo-ration with the CIA Labrsquos expertise Detailed results are then
presented to the customeralong with detailed remedi-ation information
Weinschenk says thatCenzic solutions are a perfect fit for development managersand other executives Using a dashboard customers can viewapplications in the testing phase as well as the number andtypes of vulnerabilities that are found
Weinschenk adds that testing is extremely importantbecause susceptibility to intrusion can result in major recoverycosts and regulatory penalties He also reports that Cenzic hasnt tested a single application that was not vulnerable Butdonrsquot worrymdashCenzic finds the vulnerabilities before the badguys do Put Cenzic to work for you ampamp
Keeping the Bad Guys At Bay With Cenzic Solutions
1 october 2006 the artampamp science of software testing u 21
w e b s e c u r i t y
John Weinschenk
President and Chief Executive Officer
gold sponsor
Donrsquot MissOut
On Another Issue of The
Test amp QAReport
e-newsletter
Each FREE weekly issue includes original articles that interview top thought leaders in software testingand quality trends best practices
and testQA methodologiesGet must-read articles that appear
only in this e-newsletter
Sign up atwwwstpmagcomtqa
Many tools exist that help QA departments and programmerstest applications However itrsquos also crucial that senior-
level software managers have the necessary information to inter-act with executive management and accurately assess the effec-tiveness of their development teams ldquoIf addressing software qual-ity early in the development cycle is an important priority for yourcompany Stelligent can helprdquo says Burke Cox Stelligentrsquos CEO
ldquoAs the thought leaders in early testing we use commercialand open-source technologies to assess and manage softwarequality during the development and assembly of applicationsrdquoCox says ldquoThe Quality Risk Index provides an objective meas-ure of software quality that enables an entire organization toassess quality and progressrdquo
Cox believes that Stelligent (wwwstelligentcom) transformsthe way software is developed and tested By introducing com-prehensive inspection as part of a continuous process both thedevelopment team and senior management gain crucial visibil-ity into the quality risks associated with software projects This
real-time feedback lets organ-izations manage quality longbefore products are everdelivered to the QA team forevaluation
Stelligentrsquos services suchas its Kickstart Quality RiskAssessment are used byorganizations developingsoftware using managedlanguages like C orJava providing bothstatic and dynamicanalysis on the appli-cation source andcompiled code
How effective isthe Kickstart Quality
Risk Assessment Wellone customer recentlyacquired a software
product that it believedwas of high quality based on a
due-diligence assessment of the source code andend user experience However when trying to integrate the prod-uct into its larger solution it found the product had poor toler-ance to change
Thatrsquos when the customer called Stelligent whose KickstartQuality Risk Assessment identified the problems Then its
Quality Roadmap and Seven-Point Implementation Plan identified top priorities for making programmatic changes through refactoring and introduced developer testing as an active part of the continuous integration process The customer
found that Stelligentrsquos QualityRoadmap quickly solved theproblem and then its Con-tinuous Quality program kept
them solved Case closedldquoFor many organizations our Quality Risk Index is the first
objective measure of their software quality that they have everseenrdquo Cox says ldquoThis helps the software manager by providingan independent assessment of what their quality practices cur-rently achieve When working with the executive managementteam in developing schedules approving budgets and other prod-uct management activities the software manager uses Stelligentreports as a basis for discussion For example high code com-plexity with poor test coverage might make a compelling argu-ment to delay release as opposed to simply stating that the prod-uct is just not ready for prime timerdquo
Stelligentrsquos services provide a critical and objective meas-urement of your softwarersquos quality Use this intelligence tomake it difficult to enter new defects into the source repos-itory helping your developers leverage the continuous qual-ity feedback before code even reaches the QA department
Stelligent is a subsidiary of JNetDirect which makesConvergence a solution for providing real-time visibility intosoftware quality CoView which helps your team develop accu-rate and effective JUnit tests and also high-performance JDBCdrivers But long before JNetDirect acquired Stelligent it was acustomer using the Kickstart Quality Risk Assessment servicesto help build its own products Stelligentrsquos impact was so dra-matic that JNetDirect immediately saw the value of bringingStelligentrsquos services to a broader audience
ldquoDefects will always enter the build systemrdquo Cox says ldquoOurbusiness is ensuring that they cannot live there for very longQuality may not have a material impact on initial product rev-enue but it is the most significant driver of product profitabil-ity Most companies struggle with describing quality it is mixedwith subjective measures and anecdotal evidence Our customershave a quality score they can point to when setting goals andbenchmarks For organizations interested in increasing earningslowering the costs associated with poor product quality is thebest place to startrdquo
Stelligentrsquos services start with the Kickstart Quality RiskAssessmentmdashask the company how its objective measures canhelp you too improve your software quality ampamp
Stelligent Brings ObjectivityTo Quality Measurement
t e s t i n g s e r v i c e s
Burke Cox
Chief Executive Officer
1 october 2006 the artampamp science of software testing u 23
silver sponsor
The primary mission of information technology is toincrease profits through improved business processesCompanies are constantly rethinking and struggling
with how to use IT to a competitive advantage reduce IT oper-ating and maintenance costs and reduce the total cost of own-ershiphellipall while attempting to deliver increased value
Most of these challenges are directly linked to makingsoftware workmdashwithout incurring unreasonable costs Manypeople in the industry would agree that low IT productivi-ty is the reason software development is so costly But whyare IT teams with all their expertise and hard work suf-fering from low productivity The root cause of low produc-tivity is errors made throughout the software developmentlife cycle
These errors include everything from performance errorsto security errors to misimplemented functionality to errorsthat crash an entire system They essentially stifle IT teamsrsquoability to produce working software in a reasonable time andat reasonable costs In fact if you look at virtually any ITteam you will see that its members spend about 80 percentof their time chasing and fixing bugs and only about 20percent of their time on tasks that deliver value and improvethe business This practice is far from productive
Adding to this inefficiency is the traditional softwaredevelopment approach of leaving testing till late in the devel-opment life cycle It is only then that QA does the testingnecessary to ensure that bugs are found requirements metand reliability performance and security goals achievedbefore an application is deployed into production Findingand fixing errors late in the development cycle is exponen-tially more costly time-consuming and inefficient thanaddressing them early and throughout the cycle Maintainingthis approach is a primary reason that we continue to strug-gle with quality and low productivity in the software world
Many other industries have struggled with low qualityhigh costs and low productivity as a result of human errorThe automotive industry for example recognized that althoughmistakes cannot be entirely eliminated they can be controlledThose automotive manufacturers who by taking a holisticand preventative approach to the problem making quality anintegrated focus throughout the production process and evenmodifying their production lines to prevent as many errorsas possible from ever entering the products addressed theirmost critical problems and have remained viable
The software industry still has not learned this impor-
tant lesson Many people think that error prevention is notpossible in the software industry they believe that becauseeach piece of software is different the lessons learned fromworking on one piece of software cannot be applied to oth-er pieces Instead of trying to prevent errors from enteringsoftware the industry tries to test errors out at the end ofthe development life cycle
First we build a product then we test at the end of theproduction cycle to determine whether the product worksand finally we remove any errors that testing exposesThroughout this process we cross our fingers and hope thatthe most insidious and embarrassing problems will be iden-tified before the release However a consideration of thestaggering number and impact of software errors reportedannually and their cost to the US economy suggests thatthis quality-through-end-of-cycle-testing approach is notyielding the desired results
The belief that our traditional software testing approachcan create quality software is a fundamental problem Wedonrsquot think of the whole process of building and deployingsoftware in a way that would prevent errors because we donrsquotbelieve that it can actually be done Yet this error preven-tion approach is not only possible it is necessary The increas-
ing complexity of software systems the push for faster nearcontinuous release cycles and the expanding dependencyon software for nearly every phase of business executionrequire that error prevention be addressed
If the software industry is serious about reducing theerror rate and resolving the issues that stem from errors wecanrsquot afford to continue hoping that our current approachto testing will miraculously start yielding quality softwareInstead we need to follow in the footsteps of other indus-
4 t the artampamp science of software testing 1 october 2006
Rethinking Our ApproachTo Software Quality
Testing in general must
become the responsibility
of every team member
c e o p e r s p e c t i v e
1 october 2006 the artampamp science of software testing u 5
tries and start preventing errors throughout the softwaredevelopment life cycle
Achieving Software QualityAchieving a consistently high level of quality starts with com-mitting to a practice of an end-to-end quality process Whilethere is no single silver bullet for producing reliable high-quality software there are proven steps that software devel-opment organizations can and should employ to help preventsoftware errors and improve development productivity Themost significant of these are addressed below
Establish a Quality Initiative And Group Culture Organizations need to establish a group culture that places ashared focus and importance on quality Many companies con-tinue to treat development and testing as independent disci-plines This separation of development and QA leads to manysoftware problems and inefficienciesmdashdevelopers might writecode assuming that someone in the QA department will injectquality into the software When problems occur the code justbounces back and forth between departments without anyonetaking responsibility
The manufacturing world learned long ago that it couldnot separate the responsibility of production from theresponsibility of verification and expect to achieve qualityThese responsibilities go hand in hand Organizations shouldplace development and QA under the same managementor merge the two groups completely to facilitate owner-
ship and responsibility for code quality In an establishedgroup culture developers will show that they care about
the code because caring about the code is synonymouswith caring about the group
Adopt Quality PracticesOrganizations must adopt software error preven-
tion practices from the earliest stages of devel-opment Well-known software error prevention
practices such as coding standards unit test-ing and regression testing are not regularly
followed despite their recognized value in catch-ing errors at the code level early in the cycle when it
is easiest and least costly to find and fix them These prac-tices should be an integrated part of an organizationrsquos devel-opment process Testing in general must become the respon-sibility of every team member
The trend toward service enablement of applications(eg Web services SOA) and faster shorter software releasecycles mandates that quality assurance and testing can nolonger be treated as a set event handled as an independ-ent discipline and relegated to a single proscribed phaseof the development life cycle It must become a continu-ous integrated part of the development process enabledthrough the application of known software development
quality practices
AutomateEffective adoption of software quality practicesrequires automation of as many testing tasks as possibleDevelopment organizations are increasingly burdened toproduce more code faster and with the same or fewerresources If they are to adopt software quality practicesthey will need to use software testing tools and developmentmethodologies that allow them to automate many of thesepractices and integrate them easily into their developmentprocesses Fortunately there are a growing number of prod-uct offerings that deliver this automation and integrationand that allow developers to take a ldquotest-as-you-coderdquoapproach to developmentmdashto readily create reusable testobjects and operate within a framework that facilitates cre-ating high-quality software
If their companies are to remain competitive softwareorganizations must improve their productivity by control-ling their costs while ensuring the high quality of their deliv-erables To do so requires rethinking and restructuring theway we have traditionally approached software developmentand delivery and necessitates a heightened commitment toquality throughout the software development life cycle and
Adam KolawaCo-Founder and CEO Parasoft
Everyone knows that bugs exist But at what stage of thedevelopment cycle should they be isolated and killed
Parasoft says ldquoTest early and often when bugs are easiestand least costly to find and fixrdquo Parasoftrsquos Jtest software forJava developers uses automation to make testing fast easy
and practical for developersto perform during the de-velopment life cyclersquos cod-ing stages when testing is usually the furthestthing from a program-merrsquos mind
ldquoIf you look at adeveloperrsquos resume thelast thing yoursquoll see list-ed is testing becausedevelopers just donrsquotlike to testrdquo says BrianHunt Parasoftrsquos VP of sales and actingCOO (wwwparasoftcom) ldquoHowever atsome point youhave to prove that
the software worksWe help developersvalidate what theyrsquove
built from the point of cre-ation to the completion of development That
validation starts at the desktop in the same way that spellcheckers are run against text documentsrdquo
Jtest provides Java development teams an automated unittesting and code analysis tool suite that performs compre-hensive test and analysis of Java source code exposing bugsand errors in code structure execution and design at thesource or unit level Used as a plug-in to the developerrsquos IDE(such as Eclipse WSAD Rational RAD) or integrated witha central build process Parasoft Jtest is designed to be usedby development teams in a ldquotest-as-you-coderdquo strategy tofind and eliminate errors early in the development processbefore they can infect the main application codebaseldquoUncovering errors early and at their source or root causemakes them quicker and less costly to fix and helps revealand resolve design errors that could have extended negativeimpact on an application if gone undetectedrdquo states Hunt
The newly released Jtest 80 adds new testing innovationsto help teams automatically verify the functionality of com-
plex constantly changing enterprise systems like Java EESOA and Web services reducing the risks of system down-time and security vulnerabilities At the same time teamscan find more defects with their existing resources increas-ing productivity while adhering to budget parameters
One of the most exciting new features of Jtest is itsBugDetective By automatically tracing and simulating exe-cution paths BugDetective exposes runtime defects thatwould be difficult or even impossible to find through man-ual testing or inspections With BugDetective you can nowfind diagnose and fix classes of software errors that routine-ly evade standard analysis and unit testing techniques
Jtest also lets development teams automatically generateand run tests using the popular Apache Cactus test frame-work This gives organizations early development-level defectexposure that might go unnoticed until QA deployment orproduction time when itrsquos a lot more expensive and prohib-itive to find and fix problems Another new technology isJtest Tracer which creates realistic functional JUnit test cas-es that reflect an applicationrsquos correct functional behaviorWith Jtest Tracer organizations can quickly create librariesof regression test cases that can be run to ensure that newcode changes donrsquot inadvertently break existing applicationfunctionality
ldquoThe key to reducing testing time is automationrdquo Huntdeclares ldquoJtest can even perform testing overnight to scanthe code find errors and report those errors to the devel-opers when they start working in the morning It lets themdrill straight through the results to the lines of code thatneed to be fixed It can also perform automatic functionaltests that run the code to make sure that it does what itrsquossupposed to do Because we write these tests in an openformat you can modify and extend them to meet your spe-
cific needsrdquoJtest integrates with com-
plementary Parasoft prod-ucts to provide automatedsystemwide testing solutions
for Web applications Web services and other n-tier systemsMoreover Jtest works as part of a comprehensive teamwideAutomated Error Prevention solution that provides central-ized administration and application of test practices man-agement dashboards and metrics for real-time analysis thathelp managers evaluate code compliance code readiness andteam productivity
With Jtest you catch bugs before they hatchmdashearly andoften ampamp
Parasoft Catches Bugs BeforeThey HatchmdashWith Automation
6 t the artampamp science of software testing 1 october 2006
d i a m o n d s p o n s o r
Brian Hunt
VP of Sales and Acting COO
SPONSORED WHITE PAPER
SPONSORED WHITE PAPER
SPONSORED WHITE PAPER
SPONSORED WHITE PAPER
When it comes to software testing two key words are speedand accuracymdashnot only of the application yoursquore test-
ing but also of the QA process itself Thatrsquos what teams needmdashand what TechExcel delivers in DevTest one of the three toolsin its DevSuite DevTest offers your developers test standardi-zation test reuse and powerful defect analysis capabilities whilegiving managers a birdrsquos-eye view of the entire process
ldquoDevTest is an integrated solution that allows QA teams toimprove standardization and leverage existing testing knowl-edge to carefully monitor test executionrdquo says Tieren Zhou TechExcelrsquos CEO and chief software architect ldquoDevTest accomplish-es that goal by focusing knowledge management test librarycreation planning and scheduling and test execution and analy-sis I think our approach is truly unique to the marketrdquo
DevTest is a one-stop information resource for your teamstoring all related test documentation including requirementsdocuments specifications automation scripts screen shotsand other essential components in a central repository This
ldquoknowledge viewrdquo can beused in test creation andexecution so that the testteam is always equipped to plan and execute itsassignments
A test library is built onthis knowledge founda-tion letting your teamreuse tests for new testassignments new soft-ware versions or evenentirely different soft-ware products Thesestandard test pro-cedures called test templates can belinked to supporting
documentation in theknowledge view Testtemplates can be or-
ganized and classifiedbased on products applicable
environments functional areas or any otherstructure on which a team needs to focus
Once a test library has been created QA managers and teamleads use a wizard-driven interface to assign tasks to test teamshelping them leverage everything in the test library includingprevious test assignments and defect history to aid in the plan-
ning process The test team receives its test assignment in theDevTest interface executes the items assigned to it and sub-mits defects directly from the interface into an integrated defect-tracking tool Meanwhile DevTest tracks the test results in areal-time dashboard and in presentation-quality custom reports
In addition to DevTest TechExcel (wwwtechexcelcom) offerstwo other products in the DevSuite DevTrack and DevPlan
DevTrack tracks andmanages product defectschange requests and oth-er issues facilitating team-work among users teams
and customers DevTrack also provides workflow and processautomation robust searching and reporting and point-and-click customization
The newest member of the DevSuite DevPlan is an inno-vative project-tracking tool designed exclusively for applicationlife-cycle management DevPlan unites project tracking andissue management and incorporates configurable workflowsnotifications meeting requests and process automation
Zhou explains that in order to be effective a QA organiza-tion needs both preparation and education and that TechExcelrsquostools can help
ldquoDevTest provides all of the materials needed to create andexecute test assignments in one location so that leads and man-agers have the information they need to craft effective testsand testers have the information they need to execute themrdquohe says ldquoDevTestrsquos planning wizards are a favorite amongtest managers because they allow them to query existing datato help plan their assignments and reduce guesswork In addi-tion the built-in integration with DevTrack allows testers toexecute their test assignments and log and regress defects froma single interfacerdquo
In the near future the DevSuite will support distributeddeployment allowing global development organizations toachieve the benefits of local performance even when workingon a global projectmdashstay tuned for more about this excitingdevelopment
DevTest can help both small and large organizations get ahandle on quality assurance ldquoDevTest is useful in any develop-ment environmentrdquo Zhou says ldquoItrsquos well suited for large glob-al development organizations because it provides them with ascalable real-time view of their test projects regardless of whetherthe tests are executed by their core team by an outsourced teamor by auto-testing tools However even smaller developmentteams benefit from the ability to create manage analyze andreuse their test coveragerdquo ampamp
TechExcel Takes the Guesswork Out of Quality Assurance
a p p l i c a t i o n l i f e c y c l e m a n a g e m e n t
gold sponsor
Tieren Zhou
CEO and Chief Software Architect
1 october 2006 the artampamp science of software testing u 15
The Fast amp Scalable Team Solution forDefect amp Issue Tracking bull Feature amp Change Tracking bull Task amp To-do List Tracking bull Helpdesk Ticket Tracking
OnTime is the market-leading project defect and feature management tool for agile software development and test teamsOnTime facilitates tracking analyzing and trending team-based software development efforts in an intuitive and powerful user interface A fully customizable UI powerful workflow process enforcements two-way email communications and custom reports combine to help software development teams ship software on-time
Available for Windows Web amp VSNET 20032005
800middot653middot0024
wwwaxosoftcomso f tware fo r so f tware deve lopment trade
Ship Software OnTimetrade
Only $495 for up to 5 Users bull Only $995 for up to 10 UsersFree Single-User Installations
OnTime 2006 Small Team Edition
bull For Teams up to 10 Membersbull Free Single-User Installationsbull $495 for 5-Team Membersbull $995 for 10-Team Members
OnTime 2006 Professional Edition
bull For Teams up to 10 Membersbull Free Single-User Installationsbull $495 for 5-Team Membersbull $995 for 10-Team Members
bull For Teams of 10 to 1000 Membersbull From $149 Per User
SDTimesAd_OnTime2006indd 1 62706 14151 PM
Managing the entire scope of one or more software devel-opment projects can be daunting Within the sea of
tasks features defects and test cycles among team mem-bers managers stakeholders and customers just figuringout whorsquos working on what can be a challenge Thatrsquos where
Axosoftrsquos OnTime 2006enters the picture
OnTime 2006 is adefect management sys-tem designed to help soft-ware development teamsship software on timeWith a focus on ship-ping software on timeit not only effectivelytracks and managesbugs it effectivelyaddresses the broad-er challenges andbest practices ofproject manage-ment
OnTime 2006offers maximum
flexibility for adminis-trators and users alikewith ready access from
a Windows client a Webbrowser or within Visual Studio The highly
configurable and customizable application can also be inte-grated with leading SCM packages including Perforce SCMSourceGear Vault Subversion and Visual SourceSafe
In addition to tracking defects features and tasks thesoftware governs projects with highly customizable workflowand security rules It also offers e-mail notifications and con-versation thread tracking time tracking and work loggingcustom fields and reporting audit trails and archiving
While many of Axosoftrsquos clients switch to OnTime 2006from other products utilizing its importing functionalityothers seek to upgrade from a manual process
ldquoTraditionally development tasks requests defects andother items that occur over the course of the developmentcycle have been tracked in spreadsheets providing ampleopportunity for human error and miscommunicationbetween teamsrdquo says Dan Suceava chief software architectfor Axosoft (wwwaxosoftcom)
ldquoOnTime 2006 keeps track of everything and prevents
important items from slipping through the cracksrdquo heexplains ldquoInstead of wasting developer time with trivialupdates and report requests managers can easily pull theinformation themselves OnTime 2006 frees developers frommost if not all of the process-related overhead that sur-rounds a project and it helps them focus on what they dobest building great softwarerdquo
Developers who use OnTime 2006 see a clear intuitiveview of all of their projects including all the issues defectsfeature requests milestones and tasks As they completetasks developers escalate those items to subsequent work-flow steps This can be set to automatically e-mail otherteam members when the next set of actions is ready to beperformed
OnTime 2006 allows project managers to define speci-fications workflows and security rules They can create ldquowhatifrdquo scenarios that generate predictions for completion timesworkload distribution and other milestones During the pro-jectrsquos execution project managers have access to total proj-ect visibility They know who is working on what the progressbeing made on tasks where bottlenecks are occurring defectrates and estimated completion times
After a project has been completed managers can takeadvantage of the accurate project history that continues toreside within OnTime 2006 This information can be usedfor anything from measuring various productivity rates toproviding a basis for decisions concerning future projectsThus past projects become part of a living knowledge basethat can be consulted at any time
Axosoft even offers a Customer Portal add-on for OnTime2006 that embraces customers as participants in the processof shipping software on time It provides a Web interfacetied to the OnTime database where customers can submit
bugs and other inputCustomizable security set-tings determine how muchproject information will bevisible to customers This
functionality is especially useful for consultants and ISVsduring beta-testing phases
More functionality is in the works for the productldquoOnTime 2006 is already a tool designed to enable the entiredevelopment team to ship software on timerdquo explainsSuceava ldquoWhile today it meets the needs of project man-agers developers and testers extremely well future versionswill provide further functionality for support professionalsIT directors and executivesrdquo ampamp
Axosoft Keeps DevelopmentProjects on the Fast Track
1 october 2006 the artampamp science of software testing u 17
d e f e c t t r a c k i n g
gold sponsor
Dan Suceava
Chief Software Architect
Software configuration management isnrsquot a luxurymdashforany modern software development team itrsquos a necessi-
ty But when an SCM system is as easy to use easy to admin-ister and full of productivity-enhancing features and bene-fits as Perforcersquos system itrsquoll feel like all of your developersare traveling in first class all the way
ldquoThe Perforce SCM System lets teams of local and dis-tributed developers share project files that are centrally storedand managed by the Perforce Serverrdquo says John WalkerPerforcersquos principal product consultant ldquoThe Perforce Serverhandles user requests and tracks all development activity inthe built-in Perforce database Each filersquos state informationcan be quickly discerned from any of Perforcersquos cross-platform clients This means that a user can see when a ver-sion of a particular file has been updated deleted or addedto the server Icons associated with the files indicate whethertheyrsquore currently being edited added or deleted by otherdevelopersrdquo
The Perforce SCM System(wwwperforcecom) letseach developer on your teamobtain and refresh a localprivate copy of versionedfiles by synchronizing themwith files stored in a filedepot Since all of themetadata is centralizedfile state information canbe gathered quickly andeasily Perforce efficient-ly manages both bina-ry and text files fromsource code to graph-ics to documentationThatrsquos one reasonwhy Perforce has
been adopted not onlyby pure developmentshops but also by
chip and hardwaremanufacturers that maintain
large binary assets For example Walker says the Perforce System is very pop-
ular in the game development industry where artists arerequired to create and store large numbers of large imagefiles To help all usersmdashnot just software developersmdashthePerforce software automatically renders thumbnail versions
of image files that can be viewed from within the applica-tionrsquos cross-platform graphical client P4V Regardless ofwhether yoursquore working with text or binary artifacts Perforcehandles the job with style
Perforcersquos clientserverapplication operates overany network or the In-ternet and includes itsown internal database
enginemdashsaving you money improving performance and get-ting the system up and running fast Installing the PerforceServer is simple Download the installation set from the com-panyrsquos Web site and run it Installing any one of a varietyof Perforce clients is also straightforward and the resourcesneeded to manage the installation are minimal Even largedistributed sites of 2000 or more users can be maintainedby a single administrator explains Walker
While Perforce has the basics down pat the real payoff isin the extras that the softwarersquos creators developed to makeyour own coders more efficient Perforcersquos intelligent branch-ing mechanism lets developers work on different releasebranches of a particular application in parallel Code linessupporting specific releases are clearly and visually definedin the system Once a branch is created the relationship itshares with the parent branch is tracked in the serverrsquos meta-data Since the ancestry of files is tracked between branch-es the integration history is maintained in the metadataThat makes it easy to see which changes have and havenrsquotbeen integrated between the related branchesmdashwhether yoursquoreworking in Java J2EE NET CC++ C Visual Basic orHTML
Perforce provides a basic defect-tracking system called jobsA job typically represents an enhancement request or a bugto be fixed Job definitions are customizable to support work-flow and jobs can also work with leading third-party defecttracking systems Support for Mercuryrsquos Quality Center isplanned for the next release This integration will allow usersto enter bugs in Quality Center and have them replicatedinto Perforce as jobs
ldquoThe Perforce SCM Systemrsquos high performance is the prod-uct of a streamlined architecture and closely integrated imple-mentationmdashnot expensive server or network hardwarerdquoWalker says ldquoThe Perforce Server does not require dedicat-ed hardware and client workstations never need upgradingThe systemrsquos networking capabilities arenrsquot a significant loadfor a typical LAN With little need for customization or con-figuration you can be up and running in minutesrdquo ampamp
Perforce Makes SCM FastEasy and First-Class
s o f t w a r e c o n f i g u r a t i o n m a n a g e m e n t
gold sponsor
John Walker
Principal Product Consultant
1 october 2006 the artampamp science of software testing u 19
The faster you find a security problem in the application lifecycle the better If developers catch vulnerabilities early they
can be fixed prior to pushing apps off to QA and customersFixing problems after they slip into a product can cost far morethan catching them early And of course if you donrsquot find the
defects someone else willldquoCenzic solutions provide
tremendous efficiencies andan immediate return on investmentrdquo says JohnWeinschenk Cenzicrsquospresident and CEO (wwwcenziccom) ldquoBy auto-mating the security test-ing process our cus-tomers can secure theirapplications faster andless expensively Likeperformance andfunctionality testingsecurity testingshould be automat-ed Vulnerabilitiesare an open invita-
tion to hackers AllWeb applications andinfrastructure can be
tested with Cenzic solutionsto locate security problemsrdquo
Cenzicrsquos two key security products are Cenzic Hailstorm andCenzic ClickToSecure These solutions provide the most com-prehensive and accurate tests in the industry Weinschenk saysthat while most vendors use scanning technology to focus onlyon commonly known vulnerabilities Hailstorm uses Cenzicrsquospatent-pending Stateful Assessment technology to automatepenetration testing by using a series of transactions to identifyvulnerabilities
Plus Hailstormrsquos SmartAttack Objects Library takes thesoftware beyond merely finding security holesmdashit can also helpenforce internal policies as well as bring organizations intoregulatory compliance with rules like the Gramm-Leach-BlileyAct (GLBA) SB1386 and Sarbanes-Oxley and the PaymentCard Industry (PCI) Data Security Standard and with bestpractices including SANS and OWASP in addition to manyother regulatory standards
ClickToSecure is a service that allows developers to make useof Software as a Service (SaaS) Tests are conducted remotely by
Cenzicrsquos own security expertsmdashand that means that your devel-opment team can build secure software even if it doesnrsquot haveexpertise in that area ClickToSecure accesses applicationsusing a combination of Hailstorm technology and the CenzicIntelligent Analysis (CIA) Research Lab to run assessments
In fact that CIA Lab is the foundation of the SmartAttackObjects Library Cenzic provides continuous updates based onnew vulnerabilities through the lab similar to an anti-virusmodel to help you stay ahead of the exploits and attacks Youcan use the library just as it is plus you can use CenzicrsquosSmartAttack Objects as templates that your own security archi-tects can customize for your organizationrsquos special require-ments if necessary
In addition to its software applications and services Cenzicoffers security training courses that give customers the knowl-edge and skills to use and maintain its products successfullyThe company also offers consulting services that include on-site assessment methodology and software-engineering andconsulting implementation with professional vulnerabilityconsultants who have experience in penetration testing andethical hacking
Among Cenzicrsquos many customers are Boston College DebtExchange IRIS Link and K2 Networks Boston Collegersquos devel-opment group uses Hailstorm in-house throughout its infor-mation security group to test all university Web applicationsAs they find security weaknesses developers use Hailstormreports to remediate those vulnerabilities The same group alsoaddresses its regulatory compliance issues with Hailstorm
In the case of Debt Exchange IRIS Link and K2 NetworksClickToSecure is put to work to perform security assessmentsCustomers call or fax requests to test their applications andCenzic experts test them remotely using Hailstorm in collabo-ration with the CIA Labrsquos expertise Detailed results are then
presented to the customeralong with detailed remedi-ation information
Weinschenk says thatCenzic solutions are a perfect fit for development managersand other executives Using a dashboard customers can viewapplications in the testing phase as well as the number andtypes of vulnerabilities that are found
Weinschenk adds that testing is extremely importantbecause susceptibility to intrusion can result in major recoverycosts and regulatory penalties He also reports that Cenzic hasnt tested a single application that was not vulnerable Butdonrsquot worrymdashCenzic finds the vulnerabilities before the badguys do Put Cenzic to work for you ampamp
Keeping the Bad Guys At Bay With Cenzic Solutions
1 october 2006 the artampamp science of software testing u 21
w e b s e c u r i t y
John Weinschenk
President and Chief Executive Officer
gold sponsor
Donrsquot MissOut
On Another Issue of The
Test amp QAReport
e-newsletter
Each FREE weekly issue includes original articles that interview top thought leaders in software testingand quality trends best practices
and testQA methodologiesGet must-read articles that appear
only in this e-newsletter
Sign up atwwwstpmagcomtqa
Many tools exist that help QA departments and programmerstest applications However itrsquos also crucial that senior-
level software managers have the necessary information to inter-act with executive management and accurately assess the effec-tiveness of their development teams ldquoIf addressing software qual-ity early in the development cycle is an important priority for yourcompany Stelligent can helprdquo says Burke Cox Stelligentrsquos CEO
ldquoAs the thought leaders in early testing we use commercialand open-source technologies to assess and manage softwarequality during the development and assembly of applicationsrdquoCox says ldquoThe Quality Risk Index provides an objective meas-ure of software quality that enables an entire organization toassess quality and progressrdquo
Cox believes that Stelligent (wwwstelligentcom) transformsthe way software is developed and tested By introducing com-prehensive inspection as part of a continuous process both thedevelopment team and senior management gain crucial visibil-ity into the quality risks associated with software projects This
real-time feedback lets organ-izations manage quality longbefore products are everdelivered to the QA team forevaluation
Stelligentrsquos services suchas its Kickstart Quality RiskAssessment are used byorganizations developingsoftware using managedlanguages like C orJava providing bothstatic and dynamicanalysis on the appli-cation source andcompiled code
How effective isthe Kickstart Quality
Risk Assessment Wellone customer recentlyacquired a software
product that it believedwas of high quality based on a
due-diligence assessment of the source code andend user experience However when trying to integrate the prod-uct into its larger solution it found the product had poor toler-ance to change
Thatrsquos when the customer called Stelligent whose KickstartQuality Risk Assessment identified the problems Then its
Quality Roadmap and Seven-Point Implementation Plan identified top priorities for making programmatic changes through refactoring and introduced developer testing as an active part of the continuous integration process The customer
found that Stelligentrsquos QualityRoadmap quickly solved theproblem and then its Con-tinuous Quality program kept
them solved Case closedldquoFor many organizations our Quality Risk Index is the first
objective measure of their software quality that they have everseenrdquo Cox says ldquoThis helps the software manager by providingan independent assessment of what their quality practices cur-rently achieve When working with the executive managementteam in developing schedules approving budgets and other prod-uct management activities the software manager uses Stelligentreports as a basis for discussion For example high code com-plexity with poor test coverage might make a compelling argu-ment to delay release as opposed to simply stating that the prod-uct is just not ready for prime timerdquo
Stelligentrsquos services provide a critical and objective meas-urement of your softwarersquos quality Use this intelligence tomake it difficult to enter new defects into the source repos-itory helping your developers leverage the continuous qual-ity feedback before code even reaches the QA department
Stelligent is a subsidiary of JNetDirect which makesConvergence a solution for providing real-time visibility intosoftware quality CoView which helps your team develop accu-rate and effective JUnit tests and also high-performance JDBCdrivers But long before JNetDirect acquired Stelligent it was acustomer using the Kickstart Quality Risk Assessment servicesto help build its own products Stelligentrsquos impact was so dra-matic that JNetDirect immediately saw the value of bringingStelligentrsquos services to a broader audience
ldquoDefects will always enter the build systemrdquo Cox says ldquoOurbusiness is ensuring that they cannot live there for very longQuality may not have a material impact on initial product rev-enue but it is the most significant driver of product profitabil-ity Most companies struggle with describing quality it is mixedwith subjective measures and anecdotal evidence Our customershave a quality score they can point to when setting goals andbenchmarks For organizations interested in increasing earningslowering the costs associated with poor product quality is thebest place to startrdquo
Stelligentrsquos services start with the Kickstart Quality RiskAssessmentmdashask the company how its objective measures canhelp you too improve your software quality ampamp
Stelligent Brings ObjectivityTo Quality Measurement
t e s t i n g s e r v i c e s
Burke Cox
Chief Executive Officer
1 october 2006 the artampamp science of software testing u 23
silver sponsor
1 october 2006 the artampamp science of software testing u 5
tries and start preventing errors throughout the softwaredevelopment life cycle
Achieving Software QualityAchieving a consistently high level of quality starts with com-mitting to a practice of an end-to-end quality process Whilethere is no single silver bullet for producing reliable high-quality software there are proven steps that software devel-opment organizations can and should employ to help preventsoftware errors and improve development productivity Themost significant of these are addressed below
Establish a Quality Initiative And Group Culture Organizations need to establish a group culture that places ashared focus and importance on quality Many companies con-tinue to treat development and testing as independent disci-plines This separation of development and QA leads to manysoftware problems and inefficienciesmdashdevelopers might writecode assuming that someone in the QA department will injectquality into the software When problems occur the code justbounces back and forth between departments without anyonetaking responsibility
The manufacturing world learned long ago that it couldnot separate the responsibility of production from theresponsibility of verification and expect to achieve qualityThese responsibilities go hand in hand Organizations shouldplace development and QA under the same managementor merge the two groups completely to facilitate owner-
ship and responsibility for code quality In an establishedgroup culture developers will show that they care about
the code because caring about the code is synonymouswith caring about the group
Adopt Quality PracticesOrganizations must adopt software error preven-
tion practices from the earliest stages of devel-opment Well-known software error prevention
practices such as coding standards unit test-ing and regression testing are not regularly
followed despite their recognized value in catch-ing errors at the code level early in the cycle when it
is easiest and least costly to find and fix them These prac-tices should be an integrated part of an organizationrsquos devel-opment process Testing in general must become the respon-sibility of every team member
The trend toward service enablement of applications(eg Web services SOA) and faster shorter software releasecycles mandates that quality assurance and testing can nolonger be treated as a set event handled as an independ-ent discipline and relegated to a single proscribed phaseof the development life cycle It must become a continu-ous integrated part of the development process enabledthrough the application of known software development
quality practices
AutomateEffective adoption of software quality practicesrequires automation of as many testing tasks as possibleDevelopment organizations are increasingly burdened toproduce more code faster and with the same or fewerresources If they are to adopt software quality practicesthey will need to use software testing tools and developmentmethodologies that allow them to automate many of thesepractices and integrate them easily into their developmentprocesses Fortunately there are a growing number of prod-uct offerings that deliver this automation and integrationand that allow developers to take a ldquotest-as-you-coderdquoapproach to developmentmdashto readily create reusable testobjects and operate within a framework that facilitates cre-ating high-quality software
If their companies are to remain competitive softwareorganizations must improve their productivity by control-ling their costs while ensuring the high quality of their deliv-erables To do so requires rethinking and restructuring theway we have traditionally approached software developmentand delivery and necessitates a heightened commitment toquality throughout the software development life cycle and
Adam KolawaCo-Founder and CEO Parasoft
Everyone knows that bugs exist But at what stage of thedevelopment cycle should they be isolated and killed
Parasoft says ldquoTest early and often when bugs are easiestand least costly to find and fixrdquo Parasoftrsquos Jtest software forJava developers uses automation to make testing fast easy
and practical for developersto perform during the de-velopment life cyclersquos cod-ing stages when testing is usually the furthestthing from a program-merrsquos mind
ldquoIf you look at adeveloperrsquos resume thelast thing yoursquoll see list-ed is testing becausedevelopers just donrsquotlike to testrdquo says BrianHunt Parasoftrsquos VP of sales and actingCOO (wwwparasoftcom) ldquoHowever atsome point youhave to prove that
the software worksWe help developersvalidate what theyrsquove
built from the point of cre-ation to the completion of development That
validation starts at the desktop in the same way that spellcheckers are run against text documentsrdquo
Jtest provides Java development teams an automated unittesting and code analysis tool suite that performs compre-hensive test and analysis of Java source code exposing bugsand errors in code structure execution and design at thesource or unit level Used as a plug-in to the developerrsquos IDE(such as Eclipse WSAD Rational RAD) or integrated witha central build process Parasoft Jtest is designed to be usedby development teams in a ldquotest-as-you-coderdquo strategy tofind and eliminate errors early in the development processbefore they can infect the main application codebaseldquoUncovering errors early and at their source or root causemakes them quicker and less costly to fix and helps revealand resolve design errors that could have extended negativeimpact on an application if gone undetectedrdquo states Hunt
The newly released Jtest 80 adds new testing innovationsto help teams automatically verify the functionality of com-
plex constantly changing enterprise systems like Java EESOA and Web services reducing the risks of system down-time and security vulnerabilities At the same time teamscan find more defects with their existing resources increas-ing productivity while adhering to budget parameters
One of the most exciting new features of Jtest is itsBugDetective By automatically tracing and simulating exe-cution paths BugDetective exposes runtime defects thatwould be difficult or even impossible to find through man-ual testing or inspections With BugDetective you can nowfind diagnose and fix classes of software errors that routine-ly evade standard analysis and unit testing techniques
Jtest also lets development teams automatically generateand run tests using the popular Apache Cactus test frame-work This gives organizations early development-level defectexposure that might go unnoticed until QA deployment orproduction time when itrsquos a lot more expensive and prohib-itive to find and fix problems Another new technology isJtest Tracer which creates realistic functional JUnit test cas-es that reflect an applicationrsquos correct functional behaviorWith Jtest Tracer organizations can quickly create librariesof regression test cases that can be run to ensure that newcode changes donrsquot inadvertently break existing applicationfunctionality
ldquoThe key to reducing testing time is automationrdquo Huntdeclares ldquoJtest can even perform testing overnight to scanthe code find errors and report those errors to the devel-opers when they start working in the morning It lets themdrill straight through the results to the lines of code thatneed to be fixed It can also perform automatic functionaltests that run the code to make sure that it does what itrsquossupposed to do Because we write these tests in an openformat you can modify and extend them to meet your spe-
cific needsrdquoJtest integrates with com-
plementary Parasoft prod-ucts to provide automatedsystemwide testing solutions
for Web applications Web services and other n-tier systemsMoreover Jtest works as part of a comprehensive teamwideAutomated Error Prevention solution that provides central-ized administration and application of test practices man-agement dashboards and metrics for real-time analysis thathelp managers evaluate code compliance code readiness andteam productivity
With Jtest you catch bugs before they hatchmdashearly andoften ampamp
Parasoft Catches Bugs BeforeThey HatchmdashWith Automation
6 t the artampamp science of software testing 1 october 2006
d i a m o n d s p o n s o r
Brian Hunt
VP of Sales and Acting COO
SPONSORED WHITE PAPER
SPONSORED WHITE PAPER
SPONSORED WHITE PAPER
SPONSORED WHITE PAPER
When it comes to software testing two key words are speedand accuracymdashnot only of the application yoursquore test-
ing but also of the QA process itself Thatrsquos what teams needmdashand what TechExcel delivers in DevTest one of the three toolsin its DevSuite DevTest offers your developers test standardi-zation test reuse and powerful defect analysis capabilities whilegiving managers a birdrsquos-eye view of the entire process
ldquoDevTest is an integrated solution that allows QA teams toimprove standardization and leverage existing testing knowl-edge to carefully monitor test executionrdquo says Tieren Zhou TechExcelrsquos CEO and chief software architect ldquoDevTest accomplish-es that goal by focusing knowledge management test librarycreation planning and scheduling and test execution and analy-sis I think our approach is truly unique to the marketrdquo
DevTest is a one-stop information resource for your teamstoring all related test documentation including requirementsdocuments specifications automation scripts screen shotsand other essential components in a central repository This
ldquoknowledge viewrdquo can beused in test creation andexecution so that the testteam is always equipped to plan and execute itsassignments
A test library is built onthis knowledge founda-tion letting your teamreuse tests for new testassignments new soft-ware versions or evenentirely different soft-ware products Thesestandard test pro-cedures called test templates can belinked to supporting
documentation in theknowledge view Testtemplates can be or-
ganized and classifiedbased on products applicable
environments functional areas or any otherstructure on which a team needs to focus
Once a test library has been created QA managers and teamleads use a wizard-driven interface to assign tasks to test teamshelping them leverage everything in the test library includingprevious test assignments and defect history to aid in the plan-
ning process The test team receives its test assignment in theDevTest interface executes the items assigned to it and sub-mits defects directly from the interface into an integrated defect-tracking tool Meanwhile DevTest tracks the test results in areal-time dashboard and in presentation-quality custom reports
In addition to DevTest TechExcel (wwwtechexcelcom) offerstwo other products in the DevSuite DevTrack and DevPlan
DevTrack tracks andmanages product defectschange requests and oth-er issues facilitating team-work among users teams
and customers DevTrack also provides workflow and processautomation robust searching and reporting and point-and-click customization
The newest member of the DevSuite DevPlan is an inno-vative project-tracking tool designed exclusively for applicationlife-cycle management DevPlan unites project tracking andissue management and incorporates configurable workflowsnotifications meeting requests and process automation
Zhou explains that in order to be effective a QA organiza-tion needs both preparation and education and that TechExcelrsquostools can help
ldquoDevTest provides all of the materials needed to create andexecute test assignments in one location so that leads and man-agers have the information they need to craft effective testsand testers have the information they need to execute themrdquohe says ldquoDevTestrsquos planning wizards are a favorite amongtest managers because they allow them to query existing datato help plan their assignments and reduce guesswork In addi-tion the built-in integration with DevTrack allows testers toexecute their test assignments and log and regress defects froma single interfacerdquo
In the near future the DevSuite will support distributeddeployment allowing global development organizations toachieve the benefits of local performance even when workingon a global projectmdashstay tuned for more about this excitingdevelopment
DevTest can help both small and large organizations get ahandle on quality assurance ldquoDevTest is useful in any develop-ment environmentrdquo Zhou says ldquoItrsquos well suited for large glob-al development organizations because it provides them with ascalable real-time view of their test projects regardless of whetherthe tests are executed by their core team by an outsourced teamor by auto-testing tools However even smaller developmentteams benefit from the ability to create manage analyze andreuse their test coveragerdquo ampamp
TechExcel Takes the Guesswork Out of Quality Assurance
a p p l i c a t i o n l i f e c y c l e m a n a g e m e n t
gold sponsor
Tieren Zhou
CEO and Chief Software Architect
1 october 2006 the artampamp science of software testing u 15
The Fast amp Scalable Team Solution forDefect amp Issue Tracking bull Feature amp Change Tracking bull Task amp To-do List Tracking bull Helpdesk Ticket Tracking
OnTime is the market-leading project defect and feature management tool for agile software development and test teamsOnTime facilitates tracking analyzing and trending team-based software development efforts in an intuitive and powerful user interface A fully customizable UI powerful workflow process enforcements two-way email communications and custom reports combine to help software development teams ship software on-time
Available for Windows Web amp VSNET 20032005
800middot653middot0024
wwwaxosoftcomso f tware fo r so f tware deve lopment trade
Ship Software OnTimetrade
Only $495 for up to 5 Users bull Only $995 for up to 10 UsersFree Single-User Installations
OnTime 2006 Small Team Edition
bull For Teams up to 10 Membersbull Free Single-User Installationsbull $495 for 5-Team Membersbull $995 for 10-Team Members
OnTime 2006 Professional Edition
bull For Teams up to 10 Membersbull Free Single-User Installationsbull $495 for 5-Team Membersbull $995 for 10-Team Members
bull For Teams of 10 to 1000 Membersbull From $149 Per User
SDTimesAd_OnTime2006indd 1 62706 14151 PM
Managing the entire scope of one or more software devel-opment projects can be daunting Within the sea of
tasks features defects and test cycles among team mem-bers managers stakeholders and customers just figuringout whorsquos working on what can be a challenge Thatrsquos where
Axosoftrsquos OnTime 2006enters the picture
OnTime 2006 is adefect management sys-tem designed to help soft-ware development teamsship software on timeWith a focus on ship-ping software on timeit not only effectivelytracks and managesbugs it effectivelyaddresses the broad-er challenges andbest practices ofproject manage-ment
OnTime 2006offers maximum
flexibility for adminis-trators and users alikewith ready access from
a Windows client a Webbrowser or within Visual Studio The highly
configurable and customizable application can also be inte-grated with leading SCM packages including Perforce SCMSourceGear Vault Subversion and Visual SourceSafe
In addition to tracking defects features and tasks thesoftware governs projects with highly customizable workflowand security rules It also offers e-mail notifications and con-versation thread tracking time tracking and work loggingcustom fields and reporting audit trails and archiving
While many of Axosoftrsquos clients switch to OnTime 2006from other products utilizing its importing functionalityothers seek to upgrade from a manual process
ldquoTraditionally development tasks requests defects andother items that occur over the course of the developmentcycle have been tracked in spreadsheets providing ampleopportunity for human error and miscommunicationbetween teamsrdquo says Dan Suceava chief software architectfor Axosoft (wwwaxosoftcom)
ldquoOnTime 2006 keeps track of everything and prevents
important items from slipping through the cracksrdquo heexplains ldquoInstead of wasting developer time with trivialupdates and report requests managers can easily pull theinformation themselves OnTime 2006 frees developers frommost if not all of the process-related overhead that sur-rounds a project and it helps them focus on what they dobest building great softwarerdquo
Developers who use OnTime 2006 see a clear intuitiveview of all of their projects including all the issues defectsfeature requests milestones and tasks As they completetasks developers escalate those items to subsequent work-flow steps This can be set to automatically e-mail otherteam members when the next set of actions is ready to beperformed
OnTime 2006 allows project managers to define speci-fications workflows and security rules They can create ldquowhatifrdquo scenarios that generate predictions for completion timesworkload distribution and other milestones During the pro-jectrsquos execution project managers have access to total proj-ect visibility They know who is working on what the progressbeing made on tasks where bottlenecks are occurring defectrates and estimated completion times
After a project has been completed managers can takeadvantage of the accurate project history that continues toreside within OnTime 2006 This information can be usedfor anything from measuring various productivity rates toproviding a basis for decisions concerning future projectsThus past projects become part of a living knowledge basethat can be consulted at any time
Axosoft even offers a Customer Portal add-on for OnTime2006 that embraces customers as participants in the processof shipping software on time It provides a Web interfacetied to the OnTime database where customers can submit
bugs and other inputCustomizable security set-tings determine how muchproject information will bevisible to customers This
functionality is especially useful for consultants and ISVsduring beta-testing phases
More functionality is in the works for the productldquoOnTime 2006 is already a tool designed to enable the entiredevelopment team to ship software on timerdquo explainsSuceava ldquoWhile today it meets the needs of project man-agers developers and testers extremely well future versionswill provide further functionality for support professionalsIT directors and executivesrdquo ampamp
Axosoft Keeps DevelopmentProjects on the Fast Track
1 october 2006 the artampamp science of software testing u 17
d e f e c t t r a c k i n g
gold sponsor
Dan Suceava
Chief Software Architect
Software configuration management isnrsquot a luxurymdashforany modern software development team itrsquos a necessi-
ty But when an SCM system is as easy to use easy to admin-ister and full of productivity-enhancing features and bene-fits as Perforcersquos system itrsquoll feel like all of your developersare traveling in first class all the way
ldquoThe Perforce SCM System lets teams of local and dis-tributed developers share project files that are centrally storedand managed by the Perforce Serverrdquo says John WalkerPerforcersquos principal product consultant ldquoThe Perforce Serverhandles user requests and tracks all development activity inthe built-in Perforce database Each filersquos state informationcan be quickly discerned from any of Perforcersquos cross-platform clients This means that a user can see when a ver-sion of a particular file has been updated deleted or addedto the server Icons associated with the files indicate whethertheyrsquore currently being edited added or deleted by otherdevelopersrdquo
The Perforce SCM System(wwwperforcecom) letseach developer on your teamobtain and refresh a localprivate copy of versionedfiles by synchronizing themwith files stored in a filedepot Since all of themetadata is centralizedfile state information canbe gathered quickly andeasily Perforce efficient-ly manages both bina-ry and text files fromsource code to graph-ics to documentationThatrsquos one reasonwhy Perforce has
been adopted not onlyby pure developmentshops but also by
chip and hardwaremanufacturers that maintain
large binary assets For example Walker says the Perforce System is very pop-
ular in the game development industry where artists arerequired to create and store large numbers of large imagefiles To help all usersmdashnot just software developersmdashthePerforce software automatically renders thumbnail versions
of image files that can be viewed from within the applica-tionrsquos cross-platform graphical client P4V Regardless ofwhether yoursquore working with text or binary artifacts Perforcehandles the job with style
Perforcersquos clientserverapplication operates overany network or the In-ternet and includes itsown internal database
enginemdashsaving you money improving performance and get-ting the system up and running fast Installing the PerforceServer is simple Download the installation set from the com-panyrsquos Web site and run it Installing any one of a varietyof Perforce clients is also straightforward and the resourcesneeded to manage the installation are minimal Even largedistributed sites of 2000 or more users can be maintainedby a single administrator explains Walker
While Perforce has the basics down pat the real payoff isin the extras that the softwarersquos creators developed to makeyour own coders more efficient Perforcersquos intelligent branch-ing mechanism lets developers work on different releasebranches of a particular application in parallel Code linessupporting specific releases are clearly and visually definedin the system Once a branch is created the relationship itshares with the parent branch is tracked in the serverrsquos meta-data Since the ancestry of files is tracked between branch-es the integration history is maintained in the metadataThat makes it easy to see which changes have and havenrsquotbeen integrated between the related branchesmdashwhether yoursquoreworking in Java J2EE NET CC++ C Visual Basic orHTML
Perforce provides a basic defect-tracking system called jobsA job typically represents an enhancement request or a bugto be fixed Job definitions are customizable to support work-flow and jobs can also work with leading third-party defecttracking systems Support for Mercuryrsquos Quality Center isplanned for the next release This integration will allow usersto enter bugs in Quality Center and have them replicatedinto Perforce as jobs
ldquoThe Perforce SCM Systemrsquos high performance is the prod-uct of a streamlined architecture and closely integrated imple-mentationmdashnot expensive server or network hardwarerdquoWalker says ldquoThe Perforce Server does not require dedicat-ed hardware and client workstations never need upgradingThe systemrsquos networking capabilities arenrsquot a significant loadfor a typical LAN With little need for customization or con-figuration you can be up and running in minutesrdquo ampamp
Perforce Makes SCM FastEasy and First-Class
s o f t w a r e c o n f i g u r a t i o n m a n a g e m e n t
gold sponsor
John Walker
Principal Product Consultant
1 october 2006 the artampamp science of software testing u 19
The faster you find a security problem in the application lifecycle the better If developers catch vulnerabilities early they
can be fixed prior to pushing apps off to QA and customersFixing problems after they slip into a product can cost far morethan catching them early And of course if you donrsquot find the
defects someone else willldquoCenzic solutions provide
tremendous efficiencies andan immediate return on investmentrdquo says JohnWeinschenk Cenzicrsquospresident and CEO (wwwcenziccom) ldquoBy auto-mating the security test-ing process our cus-tomers can secure theirapplications faster andless expensively Likeperformance andfunctionality testingsecurity testingshould be automat-ed Vulnerabilitiesare an open invita-
tion to hackers AllWeb applications andinfrastructure can be
tested with Cenzic solutionsto locate security problemsrdquo
Cenzicrsquos two key security products are Cenzic Hailstorm andCenzic ClickToSecure These solutions provide the most com-prehensive and accurate tests in the industry Weinschenk saysthat while most vendors use scanning technology to focus onlyon commonly known vulnerabilities Hailstorm uses Cenzicrsquospatent-pending Stateful Assessment technology to automatepenetration testing by using a series of transactions to identifyvulnerabilities
Plus Hailstormrsquos SmartAttack Objects Library takes thesoftware beyond merely finding security holesmdashit can also helpenforce internal policies as well as bring organizations intoregulatory compliance with rules like the Gramm-Leach-BlileyAct (GLBA) SB1386 and Sarbanes-Oxley and the PaymentCard Industry (PCI) Data Security Standard and with bestpractices including SANS and OWASP in addition to manyother regulatory standards
ClickToSecure is a service that allows developers to make useof Software as a Service (SaaS) Tests are conducted remotely by
Cenzicrsquos own security expertsmdashand that means that your devel-opment team can build secure software even if it doesnrsquot haveexpertise in that area ClickToSecure accesses applicationsusing a combination of Hailstorm technology and the CenzicIntelligent Analysis (CIA) Research Lab to run assessments
In fact that CIA Lab is the foundation of the SmartAttackObjects Library Cenzic provides continuous updates based onnew vulnerabilities through the lab similar to an anti-virusmodel to help you stay ahead of the exploits and attacks Youcan use the library just as it is plus you can use CenzicrsquosSmartAttack Objects as templates that your own security archi-tects can customize for your organizationrsquos special require-ments if necessary
In addition to its software applications and services Cenzicoffers security training courses that give customers the knowl-edge and skills to use and maintain its products successfullyThe company also offers consulting services that include on-site assessment methodology and software-engineering andconsulting implementation with professional vulnerabilityconsultants who have experience in penetration testing andethical hacking
Among Cenzicrsquos many customers are Boston College DebtExchange IRIS Link and K2 Networks Boston Collegersquos devel-opment group uses Hailstorm in-house throughout its infor-mation security group to test all university Web applicationsAs they find security weaknesses developers use Hailstormreports to remediate those vulnerabilities The same group alsoaddresses its regulatory compliance issues with Hailstorm
In the case of Debt Exchange IRIS Link and K2 NetworksClickToSecure is put to work to perform security assessmentsCustomers call or fax requests to test their applications andCenzic experts test them remotely using Hailstorm in collabo-ration with the CIA Labrsquos expertise Detailed results are then
presented to the customeralong with detailed remedi-ation information
Weinschenk says thatCenzic solutions are a perfect fit for development managersand other executives Using a dashboard customers can viewapplications in the testing phase as well as the number andtypes of vulnerabilities that are found
Weinschenk adds that testing is extremely importantbecause susceptibility to intrusion can result in major recoverycosts and regulatory penalties He also reports that Cenzic hasnt tested a single application that was not vulnerable Butdonrsquot worrymdashCenzic finds the vulnerabilities before the badguys do Put Cenzic to work for you ampamp
Keeping the Bad Guys At Bay With Cenzic Solutions
1 october 2006 the artampamp science of software testing u 21
w e b s e c u r i t y
John Weinschenk
President and Chief Executive Officer
gold sponsor
Donrsquot MissOut
On Another Issue of The
Test amp QAReport
e-newsletter
Each FREE weekly issue includes original articles that interview top thought leaders in software testingand quality trends best practices
and testQA methodologiesGet must-read articles that appear
only in this e-newsletter
Sign up atwwwstpmagcomtqa
Many tools exist that help QA departments and programmerstest applications However itrsquos also crucial that senior-
level software managers have the necessary information to inter-act with executive management and accurately assess the effec-tiveness of their development teams ldquoIf addressing software qual-ity early in the development cycle is an important priority for yourcompany Stelligent can helprdquo says Burke Cox Stelligentrsquos CEO
ldquoAs the thought leaders in early testing we use commercialand open-source technologies to assess and manage softwarequality during the development and assembly of applicationsrdquoCox says ldquoThe Quality Risk Index provides an objective meas-ure of software quality that enables an entire organization toassess quality and progressrdquo
Cox believes that Stelligent (wwwstelligentcom) transformsthe way software is developed and tested By introducing com-prehensive inspection as part of a continuous process both thedevelopment team and senior management gain crucial visibil-ity into the quality risks associated with software projects This
real-time feedback lets organ-izations manage quality longbefore products are everdelivered to the QA team forevaluation
Stelligentrsquos services suchas its Kickstart Quality RiskAssessment are used byorganizations developingsoftware using managedlanguages like C orJava providing bothstatic and dynamicanalysis on the appli-cation source andcompiled code
How effective isthe Kickstart Quality
Risk Assessment Wellone customer recentlyacquired a software
product that it believedwas of high quality based on a
due-diligence assessment of the source code andend user experience However when trying to integrate the prod-uct into its larger solution it found the product had poor toler-ance to change
Thatrsquos when the customer called Stelligent whose KickstartQuality Risk Assessment identified the problems Then its
Quality Roadmap and Seven-Point Implementation Plan identified top priorities for making programmatic changes through refactoring and introduced developer testing as an active part of the continuous integration process The customer
found that Stelligentrsquos QualityRoadmap quickly solved theproblem and then its Con-tinuous Quality program kept
them solved Case closedldquoFor many organizations our Quality Risk Index is the first
objective measure of their software quality that they have everseenrdquo Cox says ldquoThis helps the software manager by providingan independent assessment of what their quality practices cur-rently achieve When working with the executive managementteam in developing schedules approving budgets and other prod-uct management activities the software manager uses Stelligentreports as a basis for discussion For example high code com-plexity with poor test coverage might make a compelling argu-ment to delay release as opposed to simply stating that the prod-uct is just not ready for prime timerdquo
Stelligentrsquos services provide a critical and objective meas-urement of your softwarersquos quality Use this intelligence tomake it difficult to enter new defects into the source repos-itory helping your developers leverage the continuous qual-ity feedback before code even reaches the QA department
Stelligent is a subsidiary of JNetDirect which makesConvergence a solution for providing real-time visibility intosoftware quality CoView which helps your team develop accu-rate and effective JUnit tests and also high-performance JDBCdrivers But long before JNetDirect acquired Stelligent it was acustomer using the Kickstart Quality Risk Assessment servicesto help build its own products Stelligentrsquos impact was so dra-matic that JNetDirect immediately saw the value of bringingStelligentrsquos services to a broader audience
ldquoDefects will always enter the build systemrdquo Cox says ldquoOurbusiness is ensuring that they cannot live there for very longQuality may not have a material impact on initial product rev-enue but it is the most significant driver of product profitabil-ity Most companies struggle with describing quality it is mixedwith subjective measures and anecdotal evidence Our customershave a quality score they can point to when setting goals andbenchmarks For organizations interested in increasing earningslowering the costs associated with poor product quality is thebest place to startrdquo
Stelligentrsquos services start with the Kickstart Quality RiskAssessmentmdashask the company how its objective measures canhelp you too improve your software quality ampamp
Stelligent Brings ObjectivityTo Quality Measurement
t e s t i n g s e r v i c e s
Burke Cox
Chief Executive Officer
1 october 2006 the artampamp science of software testing u 23
silver sponsor
Everyone knows that bugs exist But at what stage of thedevelopment cycle should they be isolated and killed
Parasoft says ldquoTest early and often when bugs are easiestand least costly to find and fixrdquo Parasoftrsquos Jtest software forJava developers uses automation to make testing fast easy
and practical for developersto perform during the de-velopment life cyclersquos cod-ing stages when testing is usually the furthestthing from a program-merrsquos mind
ldquoIf you look at adeveloperrsquos resume thelast thing yoursquoll see list-ed is testing becausedevelopers just donrsquotlike to testrdquo says BrianHunt Parasoftrsquos VP of sales and actingCOO (wwwparasoftcom) ldquoHowever atsome point youhave to prove that
the software worksWe help developersvalidate what theyrsquove
built from the point of cre-ation to the completion of development That
validation starts at the desktop in the same way that spellcheckers are run against text documentsrdquo
Jtest provides Java development teams an automated unittesting and code analysis tool suite that performs compre-hensive test and analysis of Java source code exposing bugsand errors in code structure execution and design at thesource or unit level Used as a plug-in to the developerrsquos IDE(such as Eclipse WSAD Rational RAD) or integrated witha central build process Parasoft Jtest is designed to be usedby development teams in a ldquotest-as-you-coderdquo strategy tofind and eliminate errors early in the development processbefore they can infect the main application codebaseldquoUncovering errors early and at their source or root causemakes them quicker and less costly to fix and helps revealand resolve design errors that could have extended negativeimpact on an application if gone undetectedrdquo states Hunt
The newly released Jtest 80 adds new testing innovationsto help teams automatically verify the functionality of com-
plex constantly changing enterprise systems like Java EESOA and Web services reducing the risks of system down-time and security vulnerabilities At the same time teamscan find more defects with their existing resources increas-ing productivity while adhering to budget parameters
One of the most exciting new features of Jtest is itsBugDetective By automatically tracing and simulating exe-cution paths BugDetective exposes runtime defects thatwould be difficult or even impossible to find through man-ual testing or inspections With BugDetective you can nowfind diagnose and fix classes of software errors that routine-ly evade standard analysis and unit testing techniques
Jtest also lets development teams automatically generateand run tests using the popular Apache Cactus test frame-work This gives organizations early development-level defectexposure that might go unnoticed until QA deployment orproduction time when itrsquos a lot more expensive and prohib-itive to find and fix problems Another new technology isJtest Tracer which creates realistic functional JUnit test cas-es that reflect an applicationrsquos correct functional behaviorWith Jtest Tracer organizations can quickly create librariesof regression test cases that can be run to ensure that newcode changes donrsquot inadvertently break existing applicationfunctionality
ldquoThe key to reducing testing time is automationrdquo Huntdeclares ldquoJtest can even perform testing overnight to scanthe code find errors and report those errors to the devel-opers when they start working in the morning It lets themdrill straight through the results to the lines of code thatneed to be fixed It can also perform automatic functionaltests that run the code to make sure that it does what itrsquossupposed to do Because we write these tests in an openformat you can modify and extend them to meet your spe-
cific needsrdquoJtest integrates with com-
plementary Parasoft prod-ucts to provide automatedsystemwide testing solutions
for Web applications Web services and other n-tier systemsMoreover Jtest works as part of a comprehensive teamwideAutomated Error Prevention solution that provides central-ized administration and application of test practices man-agement dashboards and metrics for real-time analysis thathelp managers evaluate code compliance code readiness andteam productivity
With Jtest you catch bugs before they hatchmdashearly andoften ampamp
Parasoft Catches Bugs BeforeThey HatchmdashWith Automation
6 t the artampamp science of software testing 1 october 2006
d i a m o n d s p o n s o r
Brian Hunt
VP of Sales and Acting COO
SPONSORED WHITE PAPER
SPONSORED WHITE PAPER
SPONSORED WHITE PAPER
SPONSORED WHITE PAPER
When it comes to software testing two key words are speedand accuracymdashnot only of the application yoursquore test-
ing but also of the QA process itself Thatrsquos what teams needmdashand what TechExcel delivers in DevTest one of the three toolsin its DevSuite DevTest offers your developers test standardi-zation test reuse and powerful defect analysis capabilities whilegiving managers a birdrsquos-eye view of the entire process
ldquoDevTest is an integrated solution that allows QA teams toimprove standardization and leverage existing testing knowl-edge to carefully monitor test executionrdquo says Tieren Zhou TechExcelrsquos CEO and chief software architect ldquoDevTest accomplish-es that goal by focusing knowledge management test librarycreation planning and scheduling and test execution and analy-sis I think our approach is truly unique to the marketrdquo
DevTest is a one-stop information resource for your teamstoring all related test documentation including requirementsdocuments specifications automation scripts screen shotsand other essential components in a central repository This
ldquoknowledge viewrdquo can beused in test creation andexecution so that the testteam is always equipped to plan and execute itsassignments
A test library is built onthis knowledge founda-tion letting your teamreuse tests for new testassignments new soft-ware versions or evenentirely different soft-ware products Thesestandard test pro-cedures called test templates can belinked to supporting
documentation in theknowledge view Testtemplates can be or-
ganized and classifiedbased on products applicable
environments functional areas or any otherstructure on which a team needs to focus
Once a test library has been created QA managers and teamleads use a wizard-driven interface to assign tasks to test teamshelping them leverage everything in the test library includingprevious test assignments and defect history to aid in the plan-
ning process The test team receives its test assignment in theDevTest interface executes the items assigned to it and sub-mits defects directly from the interface into an integrated defect-tracking tool Meanwhile DevTest tracks the test results in areal-time dashboard and in presentation-quality custom reports
In addition to DevTest TechExcel (wwwtechexcelcom) offerstwo other products in the DevSuite DevTrack and DevPlan
DevTrack tracks andmanages product defectschange requests and oth-er issues facilitating team-work among users teams
and customers DevTrack also provides workflow and processautomation robust searching and reporting and point-and-click customization
The newest member of the DevSuite DevPlan is an inno-vative project-tracking tool designed exclusively for applicationlife-cycle management DevPlan unites project tracking andissue management and incorporates configurable workflowsnotifications meeting requests and process automation
Zhou explains that in order to be effective a QA organiza-tion needs both preparation and education and that TechExcelrsquostools can help
ldquoDevTest provides all of the materials needed to create andexecute test assignments in one location so that leads and man-agers have the information they need to craft effective testsand testers have the information they need to execute themrdquohe says ldquoDevTestrsquos planning wizards are a favorite amongtest managers because they allow them to query existing datato help plan their assignments and reduce guesswork In addi-tion the built-in integration with DevTrack allows testers toexecute their test assignments and log and regress defects froma single interfacerdquo
In the near future the DevSuite will support distributeddeployment allowing global development organizations toachieve the benefits of local performance even when workingon a global projectmdashstay tuned for more about this excitingdevelopment
DevTest can help both small and large organizations get ahandle on quality assurance ldquoDevTest is useful in any develop-ment environmentrdquo Zhou says ldquoItrsquos well suited for large glob-al development organizations because it provides them with ascalable real-time view of their test projects regardless of whetherthe tests are executed by their core team by an outsourced teamor by auto-testing tools However even smaller developmentteams benefit from the ability to create manage analyze andreuse their test coveragerdquo ampamp
TechExcel Takes the Guesswork Out of Quality Assurance
a p p l i c a t i o n l i f e c y c l e m a n a g e m e n t
gold sponsor
Tieren Zhou
CEO and Chief Software Architect
1 october 2006 the artampamp science of software testing u 15
The Fast amp Scalable Team Solution forDefect amp Issue Tracking bull Feature amp Change Tracking bull Task amp To-do List Tracking bull Helpdesk Ticket Tracking
OnTime is the market-leading project defect and feature management tool for agile software development and test teamsOnTime facilitates tracking analyzing and trending team-based software development efforts in an intuitive and powerful user interface A fully customizable UI powerful workflow process enforcements two-way email communications and custom reports combine to help software development teams ship software on-time
Available for Windows Web amp VSNET 20032005
800middot653middot0024
wwwaxosoftcomso f tware fo r so f tware deve lopment trade
Ship Software OnTimetrade
Only $495 for up to 5 Users bull Only $995 for up to 10 UsersFree Single-User Installations
OnTime 2006 Small Team Edition
bull For Teams up to 10 Membersbull Free Single-User Installationsbull $495 for 5-Team Membersbull $995 for 10-Team Members
OnTime 2006 Professional Edition
bull For Teams up to 10 Membersbull Free Single-User Installationsbull $495 for 5-Team Membersbull $995 for 10-Team Members
bull For Teams of 10 to 1000 Membersbull From $149 Per User
SDTimesAd_OnTime2006indd 1 62706 14151 PM
Managing the entire scope of one or more software devel-opment projects can be daunting Within the sea of
tasks features defects and test cycles among team mem-bers managers stakeholders and customers just figuringout whorsquos working on what can be a challenge Thatrsquos where
Axosoftrsquos OnTime 2006enters the picture
OnTime 2006 is adefect management sys-tem designed to help soft-ware development teamsship software on timeWith a focus on ship-ping software on timeit not only effectivelytracks and managesbugs it effectivelyaddresses the broad-er challenges andbest practices ofproject manage-ment
OnTime 2006offers maximum
flexibility for adminis-trators and users alikewith ready access from
a Windows client a Webbrowser or within Visual Studio The highly
configurable and customizable application can also be inte-grated with leading SCM packages including Perforce SCMSourceGear Vault Subversion and Visual SourceSafe
In addition to tracking defects features and tasks thesoftware governs projects with highly customizable workflowand security rules It also offers e-mail notifications and con-versation thread tracking time tracking and work loggingcustom fields and reporting audit trails and archiving
While many of Axosoftrsquos clients switch to OnTime 2006from other products utilizing its importing functionalityothers seek to upgrade from a manual process
ldquoTraditionally development tasks requests defects andother items that occur over the course of the developmentcycle have been tracked in spreadsheets providing ampleopportunity for human error and miscommunicationbetween teamsrdquo says Dan Suceava chief software architectfor Axosoft (wwwaxosoftcom)
ldquoOnTime 2006 keeps track of everything and prevents
important items from slipping through the cracksrdquo heexplains ldquoInstead of wasting developer time with trivialupdates and report requests managers can easily pull theinformation themselves OnTime 2006 frees developers frommost if not all of the process-related overhead that sur-rounds a project and it helps them focus on what they dobest building great softwarerdquo
Developers who use OnTime 2006 see a clear intuitiveview of all of their projects including all the issues defectsfeature requests milestones and tasks As they completetasks developers escalate those items to subsequent work-flow steps This can be set to automatically e-mail otherteam members when the next set of actions is ready to beperformed
OnTime 2006 allows project managers to define speci-fications workflows and security rules They can create ldquowhatifrdquo scenarios that generate predictions for completion timesworkload distribution and other milestones During the pro-jectrsquos execution project managers have access to total proj-ect visibility They know who is working on what the progressbeing made on tasks where bottlenecks are occurring defectrates and estimated completion times
After a project has been completed managers can takeadvantage of the accurate project history that continues toreside within OnTime 2006 This information can be usedfor anything from measuring various productivity rates toproviding a basis for decisions concerning future projectsThus past projects become part of a living knowledge basethat can be consulted at any time
Axosoft even offers a Customer Portal add-on for OnTime2006 that embraces customers as participants in the processof shipping software on time It provides a Web interfacetied to the OnTime database where customers can submit
bugs and other inputCustomizable security set-tings determine how muchproject information will bevisible to customers This
functionality is especially useful for consultants and ISVsduring beta-testing phases
More functionality is in the works for the productldquoOnTime 2006 is already a tool designed to enable the entiredevelopment team to ship software on timerdquo explainsSuceava ldquoWhile today it meets the needs of project man-agers developers and testers extremely well future versionswill provide further functionality for support professionalsIT directors and executivesrdquo ampamp
Axosoft Keeps DevelopmentProjects on the Fast Track
1 october 2006 the artampamp science of software testing u 17
d e f e c t t r a c k i n g
gold sponsor
Dan Suceava
Chief Software Architect
Software configuration management isnrsquot a luxurymdashforany modern software development team itrsquos a necessi-
ty But when an SCM system is as easy to use easy to admin-ister and full of productivity-enhancing features and bene-fits as Perforcersquos system itrsquoll feel like all of your developersare traveling in first class all the way
ldquoThe Perforce SCM System lets teams of local and dis-tributed developers share project files that are centrally storedand managed by the Perforce Serverrdquo says John WalkerPerforcersquos principal product consultant ldquoThe Perforce Serverhandles user requests and tracks all development activity inthe built-in Perforce database Each filersquos state informationcan be quickly discerned from any of Perforcersquos cross-platform clients This means that a user can see when a ver-sion of a particular file has been updated deleted or addedto the server Icons associated with the files indicate whethertheyrsquore currently being edited added or deleted by otherdevelopersrdquo
The Perforce SCM System(wwwperforcecom) letseach developer on your teamobtain and refresh a localprivate copy of versionedfiles by synchronizing themwith files stored in a filedepot Since all of themetadata is centralizedfile state information canbe gathered quickly andeasily Perforce efficient-ly manages both bina-ry and text files fromsource code to graph-ics to documentationThatrsquos one reasonwhy Perforce has
been adopted not onlyby pure developmentshops but also by
chip and hardwaremanufacturers that maintain
large binary assets For example Walker says the Perforce System is very pop-
ular in the game development industry where artists arerequired to create and store large numbers of large imagefiles To help all usersmdashnot just software developersmdashthePerforce software automatically renders thumbnail versions
of image files that can be viewed from within the applica-tionrsquos cross-platform graphical client P4V Regardless ofwhether yoursquore working with text or binary artifacts Perforcehandles the job with style
Perforcersquos clientserverapplication operates overany network or the In-ternet and includes itsown internal database
enginemdashsaving you money improving performance and get-ting the system up and running fast Installing the PerforceServer is simple Download the installation set from the com-panyrsquos Web site and run it Installing any one of a varietyof Perforce clients is also straightforward and the resourcesneeded to manage the installation are minimal Even largedistributed sites of 2000 or more users can be maintainedby a single administrator explains Walker
While Perforce has the basics down pat the real payoff isin the extras that the softwarersquos creators developed to makeyour own coders more efficient Perforcersquos intelligent branch-ing mechanism lets developers work on different releasebranches of a particular application in parallel Code linessupporting specific releases are clearly and visually definedin the system Once a branch is created the relationship itshares with the parent branch is tracked in the serverrsquos meta-data Since the ancestry of files is tracked between branch-es the integration history is maintained in the metadataThat makes it easy to see which changes have and havenrsquotbeen integrated between the related branchesmdashwhether yoursquoreworking in Java J2EE NET CC++ C Visual Basic orHTML
Perforce provides a basic defect-tracking system called jobsA job typically represents an enhancement request or a bugto be fixed Job definitions are customizable to support work-flow and jobs can also work with leading third-party defecttracking systems Support for Mercuryrsquos Quality Center isplanned for the next release This integration will allow usersto enter bugs in Quality Center and have them replicatedinto Perforce as jobs
ldquoThe Perforce SCM Systemrsquos high performance is the prod-uct of a streamlined architecture and closely integrated imple-mentationmdashnot expensive server or network hardwarerdquoWalker says ldquoThe Perforce Server does not require dedicat-ed hardware and client workstations never need upgradingThe systemrsquos networking capabilities arenrsquot a significant loadfor a typical LAN With little need for customization or con-figuration you can be up and running in minutesrdquo ampamp
Perforce Makes SCM FastEasy and First-Class
s o f t w a r e c o n f i g u r a t i o n m a n a g e m e n t
gold sponsor
John Walker
Principal Product Consultant
1 october 2006 the artampamp science of software testing u 19
The faster you find a security problem in the application lifecycle the better If developers catch vulnerabilities early they
can be fixed prior to pushing apps off to QA and customersFixing problems after they slip into a product can cost far morethan catching them early And of course if you donrsquot find the
defects someone else willldquoCenzic solutions provide
tremendous efficiencies andan immediate return on investmentrdquo says JohnWeinschenk Cenzicrsquospresident and CEO (wwwcenziccom) ldquoBy auto-mating the security test-ing process our cus-tomers can secure theirapplications faster andless expensively Likeperformance andfunctionality testingsecurity testingshould be automat-ed Vulnerabilitiesare an open invita-
tion to hackers AllWeb applications andinfrastructure can be
tested with Cenzic solutionsto locate security problemsrdquo
Cenzicrsquos two key security products are Cenzic Hailstorm andCenzic ClickToSecure These solutions provide the most com-prehensive and accurate tests in the industry Weinschenk saysthat while most vendors use scanning technology to focus onlyon commonly known vulnerabilities Hailstorm uses Cenzicrsquospatent-pending Stateful Assessment technology to automatepenetration testing by using a series of transactions to identifyvulnerabilities
Plus Hailstormrsquos SmartAttack Objects Library takes thesoftware beyond merely finding security holesmdashit can also helpenforce internal policies as well as bring organizations intoregulatory compliance with rules like the Gramm-Leach-BlileyAct (GLBA) SB1386 and Sarbanes-Oxley and the PaymentCard Industry (PCI) Data Security Standard and with bestpractices including SANS and OWASP in addition to manyother regulatory standards
ClickToSecure is a service that allows developers to make useof Software as a Service (SaaS) Tests are conducted remotely by
Cenzicrsquos own security expertsmdashand that means that your devel-opment team can build secure software even if it doesnrsquot haveexpertise in that area ClickToSecure accesses applicationsusing a combination of Hailstorm technology and the CenzicIntelligent Analysis (CIA) Research Lab to run assessments
In fact that CIA Lab is the foundation of the SmartAttackObjects Library Cenzic provides continuous updates based onnew vulnerabilities through the lab similar to an anti-virusmodel to help you stay ahead of the exploits and attacks Youcan use the library just as it is plus you can use CenzicrsquosSmartAttack Objects as templates that your own security archi-tects can customize for your organizationrsquos special require-ments if necessary
In addition to its software applications and services Cenzicoffers security training courses that give customers the knowl-edge and skills to use and maintain its products successfullyThe company also offers consulting services that include on-site assessment methodology and software-engineering andconsulting implementation with professional vulnerabilityconsultants who have experience in penetration testing andethical hacking
Among Cenzicrsquos many customers are Boston College DebtExchange IRIS Link and K2 Networks Boston Collegersquos devel-opment group uses Hailstorm in-house throughout its infor-mation security group to test all university Web applicationsAs they find security weaknesses developers use Hailstormreports to remediate those vulnerabilities The same group alsoaddresses its regulatory compliance issues with Hailstorm
In the case of Debt Exchange IRIS Link and K2 NetworksClickToSecure is put to work to perform security assessmentsCustomers call or fax requests to test their applications andCenzic experts test them remotely using Hailstorm in collabo-ration with the CIA Labrsquos expertise Detailed results are then
presented to the customeralong with detailed remedi-ation information
Weinschenk says thatCenzic solutions are a perfect fit for development managersand other executives Using a dashboard customers can viewapplications in the testing phase as well as the number andtypes of vulnerabilities that are found
Weinschenk adds that testing is extremely importantbecause susceptibility to intrusion can result in major recoverycosts and regulatory penalties He also reports that Cenzic hasnt tested a single application that was not vulnerable Butdonrsquot worrymdashCenzic finds the vulnerabilities before the badguys do Put Cenzic to work for you ampamp
Keeping the Bad Guys At Bay With Cenzic Solutions
1 october 2006 the artampamp science of software testing u 21
w e b s e c u r i t y
John Weinschenk
President and Chief Executive Officer
gold sponsor
Donrsquot MissOut
On Another Issue of The
Test amp QAReport
e-newsletter
Each FREE weekly issue includes original articles that interview top thought leaders in software testingand quality trends best practices
and testQA methodologiesGet must-read articles that appear
only in this e-newsletter
Sign up atwwwstpmagcomtqa
Many tools exist that help QA departments and programmerstest applications However itrsquos also crucial that senior-
level software managers have the necessary information to inter-act with executive management and accurately assess the effec-tiveness of their development teams ldquoIf addressing software qual-ity early in the development cycle is an important priority for yourcompany Stelligent can helprdquo says Burke Cox Stelligentrsquos CEO
ldquoAs the thought leaders in early testing we use commercialand open-source technologies to assess and manage softwarequality during the development and assembly of applicationsrdquoCox says ldquoThe Quality Risk Index provides an objective meas-ure of software quality that enables an entire organization toassess quality and progressrdquo
Cox believes that Stelligent (wwwstelligentcom) transformsthe way software is developed and tested By introducing com-prehensive inspection as part of a continuous process both thedevelopment team and senior management gain crucial visibil-ity into the quality risks associated with software projects This
real-time feedback lets organ-izations manage quality longbefore products are everdelivered to the QA team forevaluation
Stelligentrsquos services suchas its Kickstart Quality RiskAssessment are used byorganizations developingsoftware using managedlanguages like C orJava providing bothstatic and dynamicanalysis on the appli-cation source andcompiled code
How effective isthe Kickstart Quality
Risk Assessment Wellone customer recentlyacquired a software
product that it believedwas of high quality based on a
due-diligence assessment of the source code andend user experience However when trying to integrate the prod-uct into its larger solution it found the product had poor toler-ance to change
Thatrsquos when the customer called Stelligent whose KickstartQuality Risk Assessment identified the problems Then its
Quality Roadmap and Seven-Point Implementation Plan identified top priorities for making programmatic changes through refactoring and introduced developer testing as an active part of the continuous integration process The customer
found that Stelligentrsquos QualityRoadmap quickly solved theproblem and then its Con-tinuous Quality program kept
them solved Case closedldquoFor many organizations our Quality Risk Index is the first
objective measure of their software quality that they have everseenrdquo Cox says ldquoThis helps the software manager by providingan independent assessment of what their quality practices cur-rently achieve When working with the executive managementteam in developing schedules approving budgets and other prod-uct management activities the software manager uses Stelligentreports as a basis for discussion For example high code com-plexity with poor test coverage might make a compelling argu-ment to delay release as opposed to simply stating that the prod-uct is just not ready for prime timerdquo
Stelligentrsquos services provide a critical and objective meas-urement of your softwarersquos quality Use this intelligence tomake it difficult to enter new defects into the source repos-itory helping your developers leverage the continuous qual-ity feedback before code even reaches the QA department
Stelligent is a subsidiary of JNetDirect which makesConvergence a solution for providing real-time visibility intosoftware quality CoView which helps your team develop accu-rate and effective JUnit tests and also high-performance JDBCdrivers But long before JNetDirect acquired Stelligent it was acustomer using the Kickstart Quality Risk Assessment servicesto help build its own products Stelligentrsquos impact was so dra-matic that JNetDirect immediately saw the value of bringingStelligentrsquos services to a broader audience
ldquoDefects will always enter the build systemrdquo Cox says ldquoOurbusiness is ensuring that they cannot live there for very longQuality may not have a material impact on initial product rev-enue but it is the most significant driver of product profitabil-ity Most companies struggle with describing quality it is mixedwith subjective measures and anecdotal evidence Our customershave a quality score they can point to when setting goals andbenchmarks For organizations interested in increasing earningslowering the costs associated with poor product quality is thebest place to startrdquo
Stelligentrsquos services start with the Kickstart Quality RiskAssessmentmdashask the company how its objective measures canhelp you too improve your software quality ampamp
Stelligent Brings ObjectivityTo Quality Measurement
t e s t i n g s e r v i c e s
Burke Cox
Chief Executive Officer
1 october 2006 the artampamp science of software testing u 23
silver sponsor
SPONSORED WHITE PAPER
SPONSORED WHITE PAPER
SPONSORED WHITE PAPER
SPONSORED WHITE PAPER
When it comes to software testing two key words are speedand accuracymdashnot only of the application yoursquore test-
ing but also of the QA process itself Thatrsquos what teams needmdashand what TechExcel delivers in DevTest one of the three toolsin its DevSuite DevTest offers your developers test standardi-zation test reuse and powerful defect analysis capabilities whilegiving managers a birdrsquos-eye view of the entire process
ldquoDevTest is an integrated solution that allows QA teams toimprove standardization and leverage existing testing knowl-edge to carefully monitor test executionrdquo says Tieren Zhou TechExcelrsquos CEO and chief software architect ldquoDevTest accomplish-es that goal by focusing knowledge management test librarycreation planning and scheduling and test execution and analy-sis I think our approach is truly unique to the marketrdquo
DevTest is a one-stop information resource for your teamstoring all related test documentation including requirementsdocuments specifications automation scripts screen shotsand other essential components in a central repository This
ldquoknowledge viewrdquo can beused in test creation andexecution so that the testteam is always equipped to plan and execute itsassignments
A test library is built onthis knowledge founda-tion letting your teamreuse tests for new testassignments new soft-ware versions or evenentirely different soft-ware products Thesestandard test pro-cedures called test templates can belinked to supporting
documentation in theknowledge view Testtemplates can be or-
ganized and classifiedbased on products applicable
environments functional areas or any otherstructure on which a team needs to focus
Once a test library has been created QA managers and teamleads use a wizard-driven interface to assign tasks to test teamshelping them leverage everything in the test library includingprevious test assignments and defect history to aid in the plan-
ning process The test team receives its test assignment in theDevTest interface executes the items assigned to it and sub-mits defects directly from the interface into an integrated defect-tracking tool Meanwhile DevTest tracks the test results in areal-time dashboard and in presentation-quality custom reports
In addition to DevTest TechExcel (wwwtechexcelcom) offerstwo other products in the DevSuite DevTrack and DevPlan
DevTrack tracks andmanages product defectschange requests and oth-er issues facilitating team-work among users teams
and customers DevTrack also provides workflow and processautomation robust searching and reporting and point-and-click customization
The newest member of the DevSuite DevPlan is an inno-vative project-tracking tool designed exclusively for applicationlife-cycle management DevPlan unites project tracking andissue management and incorporates configurable workflowsnotifications meeting requests and process automation
Zhou explains that in order to be effective a QA organiza-tion needs both preparation and education and that TechExcelrsquostools can help
ldquoDevTest provides all of the materials needed to create andexecute test assignments in one location so that leads and man-agers have the information they need to craft effective testsand testers have the information they need to execute themrdquohe says ldquoDevTestrsquos planning wizards are a favorite amongtest managers because they allow them to query existing datato help plan their assignments and reduce guesswork In addi-tion the built-in integration with DevTrack allows testers toexecute their test assignments and log and regress defects froma single interfacerdquo
In the near future the DevSuite will support distributeddeployment allowing global development organizations toachieve the benefits of local performance even when workingon a global projectmdashstay tuned for more about this excitingdevelopment
DevTest can help both small and large organizations get ahandle on quality assurance ldquoDevTest is useful in any develop-ment environmentrdquo Zhou says ldquoItrsquos well suited for large glob-al development organizations because it provides them with ascalable real-time view of their test projects regardless of whetherthe tests are executed by their core team by an outsourced teamor by auto-testing tools However even smaller developmentteams benefit from the ability to create manage analyze andreuse their test coveragerdquo ampamp
TechExcel Takes the Guesswork Out of Quality Assurance
a p p l i c a t i o n l i f e c y c l e m a n a g e m e n t
gold sponsor
Tieren Zhou
CEO and Chief Software Architect
1 october 2006 the artampamp science of software testing u 15
The Fast amp Scalable Team Solution forDefect amp Issue Tracking bull Feature amp Change Tracking bull Task amp To-do List Tracking bull Helpdesk Ticket Tracking
OnTime is the market-leading project defect and feature management tool for agile software development and test teamsOnTime facilitates tracking analyzing and trending team-based software development efforts in an intuitive and powerful user interface A fully customizable UI powerful workflow process enforcements two-way email communications and custom reports combine to help software development teams ship software on-time
Available for Windows Web amp VSNET 20032005
800middot653middot0024
wwwaxosoftcomso f tware fo r so f tware deve lopment trade
Ship Software OnTimetrade
Only $495 for up to 5 Users bull Only $995 for up to 10 UsersFree Single-User Installations
OnTime 2006 Small Team Edition
bull For Teams up to 10 Membersbull Free Single-User Installationsbull $495 for 5-Team Membersbull $995 for 10-Team Members
OnTime 2006 Professional Edition
bull For Teams up to 10 Membersbull Free Single-User Installationsbull $495 for 5-Team Membersbull $995 for 10-Team Members
bull For Teams of 10 to 1000 Membersbull From $149 Per User
SDTimesAd_OnTime2006indd 1 62706 14151 PM
Managing the entire scope of one or more software devel-opment projects can be daunting Within the sea of
tasks features defects and test cycles among team mem-bers managers stakeholders and customers just figuringout whorsquos working on what can be a challenge Thatrsquos where
Axosoftrsquos OnTime 2006enters the picture
OnTime 2006 is adefect management sys-tem designed to help soft-ware development teamsship software on timeWith a focus on ship-ping software on timeit not only effectivelytracks and managesbugs it effectivelyaddresses the broad-er challenges andbest practices ofproject manage-ment
OnTime 2006offers maximum
flexibility for adminis-trators and users alikewith ready access from
a Windows client a Webbrowser or within Visual Studio The highly
configurable and customizable application can also be inte-grated with leading SCM packages including Perforce SCMSourceGear Vault Subversion and Visual SourceSafe
In addition to tracking defects features and tasks thesoftware governs projects with highly customizable workflowand security rules It also offers e-mail notifications and con-versation thread tracking time tracking and work loggingcustom fields and reporting audit trails and archiving
While many of Axosoftrsquos clients switch to OnTime 2006from other products utilizing its importing functionalityothers seek to upgrade from a manual process
ldquoTraditionally development tasks requests defects andother items that occur over the course of the developmentcycle have been tracked in spreadsheets providing ampleopportunity for human error and miscommunicationbetween teamsrdquo says Dan Suceava chief software architectfor Axosoft (wwwaxosoftcom)
ldquoOnTime 2006 keeps track of everything and prevents
important items from slipping through the cracksrdquo heexplains ldquoInstead of wasting developer time with trivialupdates and report requests managers can easily pull theinformation themselves OnTime 2006 frees developers frommost if not all of the process-related overhead that sur-rounds a project and it helps them focus on what they dobest building great softwarerdquo
Developers who use OnTime 2006 see a clear intuitiveview of all of their projects including all the issues defectsfeature requests milestones and tasks As they completetasks developers escalate those items to subsequent work-flow steps This can be set to automatically e-mail otherteam members when the next set of actions is ready to beperformed
OnTime 2006 allows project managers to define speci-fications workflows and security rules They can create ldquowhatifrdquo scenarios that generate predictions for completion timesworkload distribution and other milestones During the pro-jectrsquos execution project managers have access to total proj-ect visibility They know who is working on what the progressbeing made on tasks where bottlenecks are occurring defectrates and estimated completion times
After a project has been completed managers can takeadvantage of the accurate project history that continues toreside within OnTime 2006 This information can be usedfor anything from measuring various productivity rates toproviding a basis for decisions concerning future projectsThus past projects become part of a living knowledge basethat can be consulted at any time
Axosoft even offers a Customer Portal add-on for OnTime2006 that embraces customers as participants in the processof shipping software on time It provides a Web interfacetied to the OnTime database where customers can submit
bugs and other inputCustomizable security set-tings determine how muchproject information will bevisible to customers This
functionality is especially useful for consultants and ISVsduring beta-testing phases
More functionality is in the works for the productldquoOnTime 2006 is already a tool designed to enable the entiredevelopment team to ship software on timerdquo explainsSuceava ldquoWhile today it meets the needs of project man-agers developers and testers extremely well future versionswill provide further functionality for support professionalsIT directors and executivesrdquo ampamp
Axosoft Keeps DevelopmentProjects on the Fast Track
1 october 2006 the artampamp science of software testing u 17
d e f e c t t r a c k i n g
gold sponsor
Dan Suceava
Chief Software Architect
Software configuration management isnrsquot a luxurymdashforany modern software development team itrsquos a necessi-
ty But when an SCM system is as easy to use easy to admin-ister and full of productivity-enhancing features and bene-fits as Perforcersquos system itrsquoll feel like all of your developersare traveling in first class all the way
ldquoThe Perforce SCM System lets teams of local and dis-tributed developers share project files that are centrally storedand managed by the Perforce Serverrdquo says John WalkerPerforcersquos principal product consultant ldquoThe Perforce Serverhandles user requests and tracks all development activity inthe built-in Perforce database Each filersquos state informationcan be quickly discerned from any of Perforcersquos cross-platform clients This means that a user can see when a ver-sion of a particular file has been updated deleted or addedto the server Icons associated with the files indicate whethertheyrsquore currently being edited added or deleted by otherdevelopersrdquo
The Perforce SCM System(wwwperforcecom) letseach developer on your teamobtain and refresh a localprivate copy of versionedfiles by synchronizing themwith files stored in a filedepot Since all of themetadata is centralizedfile state information canbe gathered quickly andeasily Perforce efficient-ly manages both bina-ry and text files fromsource code to graph-ics to documentationThatrsquos one reasonwhy Perforce has
been adopted not onlyby pure developmentshops but also by
chip and hardwaremanufacturers that maintain
large binary assets For example Walker says the Perforce System is very pop-
ular in the game development industry where artists arerequired to create and store large numbers of large imagefiles To help all usersmdashnot just software developersmdashthePerforce software automatically renders thumbnail versions
of image files that can be viewed from within the applica-tionrsquos cross-platform graphical client P4V Regardless ofwhether yoursquore working with text or binary artifacts Perforcehandles the job with style
Perforcersquos clientserverapplication operates overany network or the In-ternet and includes itsown internal database
enginemdashsaving you money improving performance and get-ting the system up and running fast Installing the PerforceServer is simple Download the installation set from the com-panyrsquos Web site and run it Installing any one of a varietyof Perforce clients is also straightforward and the resourcesneeded to manage the installation are minimal Even largedistributed sites of 2000 or more users can be maintainedby a single administrator explains Walker
While Perforce has the basics down pat the real payoff isin the extras that the softwarersquos creators developed to makeyour own coders more efficient Perforcersquos intelligent branch-ing mechanism lets developers work on different releasebranches of a particular application in parallel Code linessupporting specific releases are clearly and visually definedin the system Once a branch is created the relationship itshares with the parent branch is tracked in the serverrsquos meta-data Since the ancestry of files is tracked between branch-es the integration history is maintained in the metadataThat makes it easy to see which changes have and havenrsquotbeen integrated between the related branchesmdashwhether yoursquoreworking in Java J2EE NET CC++ C Visual Basic orHTML
Perforce provides a basic defect-tracking system called jobsA job typically represents an enhancement request or a bugto be fixed Job definitions are customizable to support work-flow and jobs can also work with leading third-party defecttracking systems Support for Mercuryrsquos Quality Center isplanned for the next release This integration will allow usersto enter bugs in Quality Center and have them replicatedinto Perforce as jobs
ldquoThe Perforce SCM Systemrsquos high performance is the prod-uct of a streamlined architecture and closely integrated imple-mentationmdashnot expensive server or network hardwarerdquoWalker says ldquoThe Perforce Server does not require dedicat-ed hardware and client workstations never need upgradingThe systemrsquos networking capabilities arenrsquot a significant loadfor a typical LAN With little need for customization or con-figuration you can be up and running in minutesrdquo ampamp
Perforce Makes SCM FastEasy and First-Class
s o f t w a r e c o n f i g u r a t i o n m a n a g e m e n t
gold sponsor
John Walker
Principal Product Consultant
1 october 2006 the artampamp science of software testing u 19
The faster you find a security problem in the application lifecycle the better If developers catch vulnerabilities early they
can be fixed prior to pushing apps off to QA and customersFixing problems after they slip into a product can cost far morethan catching them early And of course if you donrsquot find the
defects someone else willldquoCenzic solutions provide
tremendous efficiencies andan immediate return on investmentrdquo says JohnWeinschenk Cenzicrsquospresident and CEO (wwwcenziccom) ldquoBy auto-mating the security test-ing process our cus-tomers can secure theirapplications faster andless expensively Likeperformance andfunctionality testingsecurity testingshould be automat-ed Vulnerabilitiesare an open invita-
tion to hackers AllWeb applications andinfrastructure can be
tested with Cenzic solutionsto locate security problemsrdquo
Cenzicrsquos two key security products are Cenzic Hailstorm andCenzic ClickToSecure These solutions provide the most com-prehensive and accurate tests in the industry Weinschenk saysthat while most vendors use scanning technology to focus onlyon commonly known vulnerabilities Hailstorm uses Cenzicrsquospatent-pending Stateful Assessment technology to automatepenetration testing by using a series of transactions to identifyvulnerabilities
Plus Hailstormrsquos SmartAttack Objects Library takes thesoftware beyond merely finding security holesmdashit can also helpenforce internal policies as well as bring organizations intoregulatory compliance with rules like the Gramm-Leach-BlileyAct (GLBA) SB1386 and Sarbanes-Oxley and the PaymentCard Industry (PCI) Data Security Standard and with bestpractices including SANS and OWASP in addition to manyother regulatory standards
ClickToSecure is a service that allows developers to make useof Software as a Service (SaaS) Tests are conducted remotely by
Cenzicrsquos own security expertsmdashand that means that your devel-opment team can build secure software even if it doesnrsquot haveexpertise in that area ClickToSecure accesses applicationsusing a combination of Hailstorm technology and the CenzicIntelligent Analysis (CIA) Research Lab to run assessments
In fact that CIA Lab is the foundation of the SmartAttackObjects Library Cenzic provides continuous updates based onnew vulnerabilities through the lab similar to an anti-virusmodel to help you stay ahead of the exploits and attacks Youcan use the library just as it is plus you can use CenzicrsquosSmartAttack Objects as templates that your own security archi-tects can customize for your organizationrsquos special require-ments if necessary
In addition to its software applications and services Cenzicoffers security training courses that give customers the knowl-edge and skills to use and maintain its products successfullyThe company also offers consulting services that include on-site assessment methodology and software-engineering andconsulting implementation with professional vulnerabilityconsultants who have experience in penetration testing andethical hacking
Among Cenzicrsquos many customers are Boston College DebtExchange IRIS Link and K2 Networks Boston Collegersquos devel-opment group uses Hailstorm in-house throughout its infor-mation security group to test all university Web applicationsAs they find security weaknesses developers use Hailstormreports to remediate those vulnerabilities The same group alsoaddresses its regulatory compliance issues with Hailstorm
In the case of Debt Exchange IRIS Link and K2 NetworksClickToSecure is put to work to perform security assessmentsCustomers call or fax requests to test their applications andCenzic experts test them remotely using Hailstorm in collabo-ration with the CIA Labrsquos expertise Detailed results are then
presented to the customeralong with detailed remedi-ation information
Weinschenk says thatCenzic solutions are a perfect fit for development managersand other executives Using a dashboard customers can viewapplications in the testing phase as well as the number andtypes of vulnerabilities that are found
Weinschenk adds that testing is extremely importantbecause susceptibility to intrusion can result in major recoverycosts and regulatory penalties He also reports that Cenzic hasnt tested a single application that was not vulnerable Butdonrsquot worrymdashCenzic finds the vulnerabilities before the badguys do Put Cenzic to work for you ampamp
Keeping the Bad Guys At Bay With Cenzic Solutions
1 october 2006 the artampamp science of software testing u 21
w e b s e c u r i t y
John Weinschenk
President and Chief Executive Officer
gold sponsor
Donrsquot MissOut
On Another Issue of The
Test amp QAReport
e-newsletter
Each FREE weekly issue includes original articles that interview top thought leaders in software testingand quality trends best practices
and testQA methodologiesGet must-read articles that appear
only in this e-newsletter
Sign up atwwwstpmagcomtqa
Many tools exist that help QA departments and programmerstest applications However itrsquos also crucial that senior-
level software managers have the necessary information to inter-act with executive management and accurately assess the effec-tiveness of their development teams ldquoIf addressing software qual-ity early in the development cycle is an important priority for yourcompany Stelligent can helprdquo says Burke Cox Stelligentrsquos CEO
ldquoAs the thought leaders in early testing we use commercialand open-source technologies to assess and manage softwarequality during the development and assembly of applicationsrdquoCox says ldquoThe Quality Risk Index provides an objective meas-ure of software quality that enables an entire organization toassess quality and progressrdquo
Cox believes that Stelligent (wwwstelligentcom) transformsthe way software is developed and tested By introducing com-prehensive inspection as part of a continuous process both thedevelopment team and senior management gain crucial visibil-ity into the quality risks associated with software projects This
real-time feedback lets organ-izations manage quality longbefore products are everdelivered to the QA team forevaluation
Stelligentrsquos services suchas its Kickstart Quality RiskAssessment are used byorganizations developingsoftware using managedlanguages like C orJava providing bothstatic and dynamicanalysis on the appli-cation source andcompiled code
How effective isthe Kickstart Quality
Risk Assessment Wellone customer recentlyacquired a software
product that it believedwas of high quality based on a
due-diligence assessment of the source code andend user experience However when trying to integrate the prod-uct into its larger solution it found the product had poor toler-ance to change
Thatrsquos when the customer called Stelligent whose KickstartQuality Risk Assessment identified the problems Then its
Quality Roadmap and Seven-Point Implementation Plan identified top priorities for making programmatic changes through refactoring and introduced developer testing as an active part of the continuous integration process The customer
found that Stelligentrsquos QualityRoadmap quickly solved theproblem and then its Con-tinuous Quality program kept
them solved Case closedldquoFor many organizations our Quality Risk Index is the first
objective measure of their software quality that they have everseenrdquo Cox says ldquoThis helps the software manager by providingan independent assessment of what their quality practices cur-rently achieve When working with the executive managementteam in developing schedules approving budgets and other prod-uct management activities the software manager uses Stelligentreports as a basis for discussion For example high code com-plexity with poor test coverage might make a compelling argu-ment to delay release as opposed to simply stating that the prod-uct is just not ready for prime timerdquo
Stelligentrsquos services provide a critical and objective meas-urement of your softwarersquos quality Use this intelligence tomake it difficult to enter new defects into the source repos-itory helping your developers leverage the continuous qual-ity feedback before code even reaches the QA department
Stelligent is a subsidiary of JNetDirect which makesConvergence a solution for providing real-time visibility intosoftware quality CoView which helps your team develop accu-rate and effective JUnit tests and also high-performance JDBCdrivers But long before JNetDirect acquired Stelligent it was acustomer using the Kickstart Quality Risk Assessment servicesto help build its own products Stelligentrsquos impact was so dra-matic that JNetDirect immediately saw the value of bringingStelligentrsquos services to a broader audience
ldquoDefects will always enter the build systemrdquo Cox says ldquoOurbusiness is ensuring that they cannot live there for very longQuality may not have a material impact on initial product rev-enue but it is the most significant driver of product profitabil-ity Most companies struggle with describing quality it is mixedwith subjective measures and anecdotal evidence Our customershave a quality score they can point to when setting goals andbenchmarks For organizations interested in increasing earningslowering the costs associated with poor product quality is thebest place to startrdquo
Stelligentrsquos services start with the Kickstart Quality RiskAssessmentmdashask the company how its objective measures canhelp you too improve your software quality ampamp
Stelligent Brings ObjectivityTo Quality Measurement
t e s t i n g s e r v i c e s
Burke Cox
Chief Executive Officer
1 october 2006 the artampamp science of software testing u 23
silver sponsor
SPONSORED WHITE PAPER
SPONSORED WHITE PAPER
SPONSORED WHITE PAPER
When it comes to software testing two key words are speedand accuracymdashnot only of the application yoursquore test-
ing but also of the QA process itself Thatrsquos what teams needmdashand what TechExcel delivers in DevTest one of the three toolsin its DevSuite DevTest offers your developers test standardi-zation test reuse and powerful defect analysis capabilities whilegiving managers a birdrsquos-eye view of the entire process
ldquoDevTest is an integrated solution that allows QA teams toimprove standardization and leverage existing testing knowl-edge to carefully monitor test executionrdquo says Tieren Zhou TechExcelrsquos CEO and chief software architect ldquoDevTest accomplish-es that goal by focusing knowledge management test librarycreation planning and scheduling and test execution and analy-sis I think our approach is truly unique to the marketrdquo
DevTest is a one-stop information resource for your teamstoring all related test documentation including requirementsdocuments specifications automation scripts screen shotsand other essential components in a central repository This
ldquoknowledge viewrdquo can beused in test creation andexecution so that the testteam is always equipped to plan and execute itsassignments
A test library is built onthis knowledge founda-tion letting your teamreuse tests for new testassignments new soft-ware versions or evenentirely different soft-ware products Thesestandard test pro-cedures called test templates can belinked to supporting
documentation in theknowledge view Testtemplates can be or-
ganized and classifiedbased on products applicable
environments functional areas or any otherstructure on which a team needs to focus
Once a test library has been created QA managers and teamleads use a wizard-driven interface to assign tasks to test teamshelping them leverage everything in the test library includingprevious test assignments and defect history to aid in the plan-
ning process The test team receives its test assignment in theDevTest interface executes the items assigned to it and sub-mits defects directly from the interface into an integrated defect-tracking tool Meanwhile DevTest tracks the test results in areal-time dashboard and in presentation-quality custom reports
In addition to DevTest TechExcel (wwwtechexcelcom) offerstwo other products in the DevSuite DevTrack and DevPlan
DevTrack tracks andmanages product defectschange requests and oth-er issues facilitating team-work among users teams
and customers DevTrack also provides workflow and processautomation robust searching and reporting and point-and-click customization
The newest member of the DevSuite DevPlan is an inno-vative project-tracking tool designed exclusively for applicationlife-cycle management DevPlan unites project tracking andissue management and incorporates configurable workflowsnotifications meeting requests and process automation
Zhou explains that in order to be effective a QA organiza-tion needs both preparation and education and that TechExcelrsquostools can help
ldquoDevTest provides all of the materials needed to create andexecute test assignments in one location so that leads and man-agers have the information they need to craft effective testsand testers have the information they need to execute themrdquohe says ldquoDevTestrsquos planning wizards are a favorite amongtest managers because they allow them to query existing datato help plan their assignments and reduce guesswork In addi-tion the built-in integration with DevTrack allows testers toexecute their test assignments and log and regress defects froma single interfacerdquo
In the near future the DevSuite will support distributeddeployment allowing global development organizations toachieve the benefits of local performance even when workingon a global projectmdashstay tuned for more about this excitingdevelopment
DevTest can help both small and large organizations get ahandle on quality assurance ldquoDevTest is useful in any develop-ment environmentrdquo Zhou says ldquoItrsquos well suited for large glob-al development organizations because it provides them with ascalable real-time view of their test projects regardless of whetherthe tests are executed by their core team by an outsourced teamor by auto-testing tools However even smaller developmentteams benefit from the ability to create manage analyze andreuse their test coveragerdquo ampamp
TechExcel Takes the Guesswork Out of Quality Assurance
a p p l i c a t i o n l i f e c y c l e m a n a g e m e n t
gold sponsor
Tieren Zhou
CEO and Chief Software Architect
1 october 2006 the artampamp science of software testing u 15
The Fast amp Scalable Team Solution forDefect amp Issue Tracking bull Feature amp Change Tracking bull Task amp To-do List Tracking bull Helpdesk Ticket Tracking
OnTime is the market-leading project defect and feature management tool for agile software development and test teamsOnTime facilitates tracking analyzing and trending team-based software development efforts in an intuitive and powerful user interface A fully customizable UI powerful workflow process enforcements two-way email communications and custom reports combine to help software development teams ship software on-time
Available for Windows Web amp VSNET 20032005
800middot653middot0024
wwwaxosoftcomso f tware fo r so f tware deve lopment trade
Ship Software OnTimetrade
Only $495 for up to 5 Users bull Only $995 for up to 10 UsersFree Single-User Installations
OnTime 2006 Small Team Edition
bull For Teams up to 10 Membersbull Free Single-User Installationsbull $495 for 5-Team Membersbull $995 for 10-Team Members
OnTime 2006 Professional Edition
bull For Teams up to 10 Membersbull Free Single-User Installationsbull $495 for 5-Team Membersbull $995 for 10-Team Members
bull For Teams of 10 to 1000 Membersbull From $149 Per User
SDTimesAd_OnTime2006indd 1 62706 14151 PM
Managing the entire scope of one or more software devel-opment projects can be daunting Within the sea of
tasks features defects and test cycles among team mem-bers managers stakeholders and customers just figuringout whorsquos working on what can be a challenge Thatrsquos where
Axosoftrsquos OnTime 2006enters the picture
OnTime 2006 is adefect management sys-tem designed to help soft-ware development teamsship software on timeWith a focus on ship-ping software on timeit not only effectivelytracks and managesbugs it effectivelyaddresses the broad-er challenges andbest practices ofproject manage-ment
OnTime 2006offers maximum
flexibility for adminis-trators and users alikewith ready access from
a Windows client a Webbrowser or within Visual Studio The highly
configurable and customizable application can also be inte-grated with leading SCM packages including Perforce SCMSourceGear Vault Subversion and Visual SourceSafe
In addition to tracking defects features and tasks thesoftware governs projects with highly customizable workflowand security rules It also offers e-mail notifications and con-versation thread tracking time tracking and work loggingcustom fields and reporting audit trails and archiving
While many of Axosoftrsquos clients switch to OnTime 2006from other products utilizing its importing functionalityothers seek to upgrade from a manual process
ldquoTraditionally development tasks requests defects andother items that occur over the course of the developmentcycle have been tracked in spreadsheets providing ampleopportunity for human error and miscommunicationbetween teamsrdquo says Dan Suceava chief software architectfor Axosoft (wwwaxosoftcom)
ldquoOnTime 2006 keeps track of everything and prevents
important items from slipping through the cracksrdquo heexplains ldquoInstead of wasting developer time with trivialupdates and report requests managers can easily pull theinformation themselves OnTime 2006 frees developers frommost if not all of the process-related overhead that sur-rounds a project and it helps them focus on what they dobest building great softwarerdquo
Developers who use OnTime 2006 see a clear intuitiveview of all of their projects including all the issues defectsfeature requests milestones and tasks As they completetasks developers escalate those items to subsequent work-flow steps This can be set to automatically e-mail otherteam members when the next set of actions is ready to beperformed
OnTime 2006 allows project managers to define speci-fications workflows and security rules They can create ldquowhatifrdquo scenarios that generate predictions for completion timesworkload distribution and other milestones During the pro-jectrsquos execution project managers have access to total proj-ect visibility They know who is working on what the progressbeing made on tasks where bottlenecks are occurring defectrates and estimated completion times
After a project has been completed managers can takeadvantage of the accurate project history that continues toreside within OnTime 2006 This information can be usedfor anything from measuring various productivity rates toproviding a basis for decisions concerning future projectsThus past projects become part of a living knowledge basethat can be consulted at any time
Axosoft even offers a Customer Portal add-on for OnTime2006 that embraces customers as participants in the processof shipping software on time It provides a Web interfacetied to the OnTime database where customers can submit
bugs and other inputCustomizable security set-tings determine how muchproject information will bevisible to customers This
functionality is especially useful for consultants and ISVsduring beta-testing phases
More functionality is in the works for the productldquoOnTime 2006 is already a tool designed to enable the entiredevelopment team to ship software on timerdquo explainsSuceava ldquoWhile today it meets the needs of project man-agers developers and testers extremely well future versionswill provide further functionality for support professionalsIT directors and executivesrdquo ampamp
Axosoft Keeps DevelopmentProjects on the Fast Track
1 october 2006 the artampamp science of software testing u 17
d e f e c t t r a c k i n g
gold sponsor
Dan Suceava
Chief Software Architect
Software configuration management isnrsquot a luxurymdashforany modern software development team itrsquos a necessi-
ty But when an SCM system is as easy to use easy to admin-ister and full of productivity-enhancing features and bene-fits as Perforcersquos system itrsquoll feel like all of your developersare traveling in first class all the way
ldquoThe Perforce SCM System lets teams of local and dis-tributed developers share project files that are centrally storedand managed by the Perforce Serverrdquo says John WalkerPerforcersquos principal product consultant ldquoThe Perforce Serverhandles user requests and tracks all development activity inthe built-in Perforce database Each filersquos state informationcan be quickly discerned from any of Perforcersquos cross-platform clients This means that a user can see when a ver-sion of a particular file has been updated deleted or addedto the server Icons associated with the files indicate whethertheyrsquore currently being edited added or deleted by otherdevelopersrdquo
The Perforce SCM System(wwwperforcecom) letseach developer on your teamobtain and refresh a localprivate copy of versionedfiles by synchronizing themwith files stored in a filedepot Since all of themetadata is centralizedfile state information canbe gathered quickly andeasily Perforce efficient-ly manages both bina-ry and text files fromsource code to graph-ics to documentationThatrsquos one reasonwhy Perforce has
been adopted not onlyby pure developmentshops but also by
chip and hardwaremanufacturers that maintain
large binary assets For example Walker says the Perforce System is very pop-
ular in the game development industry where artists arerequired to create and store large numbers of large imagefiles To help all usersmdashnot just software developersmdashthePerforce software automatically renders thumbnail versions
of image files that can be viewed from within the applica-tionrsquos cross-platform graphical client P4V Regardless ofwhether yoursquore working with text or binary artifacts Perforcehandles the job with style
Perforcersquos clientserverapplication operates overany network or the In-ternet and includes itsown internal database
enginemdashsaving you money improving performance and get-ting the system up and running fast Installing the PerforceServer is simple Download the installation set from the com-panyrsquos Web site and run it Installing any one of a varietyof Perforce clients is also straightforward and the resourcesneeded to manage the installation are minimal Even largedistributed sites of 2000 or more users can be maintainedby a single administrator explains Walker
While Perforce has the basics down pat the real payoff isin the extras that the softwarersquos creators developed to makeyour own coders more efficient Perforcersquos intelligent branch-ing mechanism lets developers work on different releasebranches of a particular application in parallel Code linessupporting specific releases are clearly and visually definedin the system Once a branch is created the relationship itshares with the parent branch is tracked in the serverrsquos meta-data Since the ancestry of files is tracked between branch-es the integration history is maintained in the metadataThat makes it easy to see which changes have and havenrsquotbeen integrated between the related branchesmdashwhether yoursquoreworking in Java J2EE NET CC++ C Visual Basic orHTML
Perforce provides a basic defect-tracking system called jobsA job typically represents an enhancement request or a bugto be fixed Job definitions are customizable to support work-flow and jobs can also work with leading third-party defecttracking systems Support for Mercuryrsquos Quality Center isplanned for the next release This integration will allow usersto enter bugs in Quality Center and have them replicatedinto Perforce as jobs
ldquoThe Perforce SCM Systemrsquos high performance is the prod-uct of a streamlined architecture and closely integrated imple-mentationmdashnot expensive server or network hardwarerdquoWalker says ldquoThe Perforce Server does not require dedicat-ed hardware and client workstations never need upgradingThe systemrsquos networking capabilities arenrsquot a significant loadfor a typical LAN With little need for customization or con-figuration you can be up and running in minutesrdquo ampamp
Perforce Makes SCM FastEasy and First-Class
s o f t w a r e c o n f i g u r a t i o n m a n a g e m e n t
gold sponsor
John Walker
Principal Product Consultant
1 october 2006 the artampamp science of software testing u 19
The faster you find a security problem in the application lifecycle the better If developers catch vulnerabilities early they
can be fixed prior to pushing apps off to QA and customersFixing problems after they slip into a product can cost far morethan catching them early And of course if you donrsquot find the
defects someone else willldquoCenzic solutions provide
tremendous efficiencies andan immediate return on investmentrdquo says JohnWeinschenk Cenzicrsquospresident and CEO (wwwcenziccom) ldquoBy auto-mating the security test-ing process our cus-tomers can secure theirapplications faster andless expensively Likeperformance andfunctionality testingsecurity testingshould be automat-ed Vulnerabilitiesare an open invita-
tion to hackers AllWeb applications andinfrastructure can be
tested with Cenzic solutionsto locate security problemsrdquo
Cenzicrsquos two key security products are Cenzic Hailstorm andCenzic ClickToSecure These solutions provide the most com-prehensive and accurate tests in the industry Weinschenk saysthat while most vendors use scanning technology to focus onlyon commonly known vulnerabilities Hailstorm uses Cenzicrsquospatent-pending Stateful Assessment technology to automatepenetration testing by using a series of transactions to identifyvulnerabilities
Plus Hailstormrsquos SmartAttack Objects Library takes thesoftware beyond merely finding security holesmdashit can also helpenforce internal policies as well as bring organizations intoregulatory compliance with rules like the Gramm-Leach-BlileyAct (GLBA) SB1386 and Sarbanes-Oxley and the PaymentCard Industry (PCI) Data Security Standard and with bestpractices including SANS and OWASP in addition to manyother regulatory standards
ClickToSecure is a service that allows developers to make useof Software as a Service (SaaS) Tests are conducted remotely by
Cenzicrsquos own security expertsmdashand that means that your devel-opment team can build secure software even if it doesnrsquot haveexpertise in that area ClickToSecure accesses applicationsusing a combination of Hailstorm technology and the CenzicIntelligent Analysis (CIA) Research Lab to run assessments
In fact that CIA Lab is the foundation of the SmartAttackObjects Library Cenzic provides continuous updates based onnew vulnerabilities through the lab similar to an anti-virusmodel to help you stay ahead of the exploits and attacks Youcan use the library just as it is plus you can use CenzicrsquosSmartAttack Objects as templates that your own security archi-tects can customize for your organizationrsquos special require-ments if necessary
In addition to its software applications and services Cenzicoffers security training courses that give customers the knowl-edge and skills to use and maintain its products successfullyThe company also offers consulting services that include on-site assessment methodology and software-engineering andconsulting implementation with professional vulnerabilityconsultants who have experience in penetration testing andethical hacking
Among Cenzicrsquos many customers are Boston College DebtExchange IRIS Link and K2 Networks Boston Collegersquos devel-opment group uses Hailstorm in-house throughout its infor-mation security group to test all university Web applicationsAs they find security weaknesses developers use Hailstormreports to remediate those vulnerabilities The same group alsoaddresses its regulatory compliance issues with Hailstorm
In the case of Debt Exchange IRIS Link and K2 NetworksClickToSecure is put to work to perform security assessmentsCustomers call or fax requests to test their applications andCenzic experts test them remotely using Hailstorm in collabo-ration with the CIA Labrsquos expertise Detailed results are then
presented to the customeralong with detailed remedi-ation information
Weinschenk says thatCenzic solutions are a perfect fit for development managersand other executives Using a dashboard customers can viewapplications in the testing phase as well as the number andtypes of vulnerabilities that are found
Weinschenk adds that testing is extremely importantbecause susceptibility to intrusion can result in major recoverycosts and regulatory penalties He also reports that Cenzic hasnt tested a single application that was not vulnerable Butdonrsquot worrymdashCenzic finds the vulnerabilities before the badguys do Put Cenzic to work for you ampamp
Keeping the Bad Guys At Bay With Cenzic Solutions
1 october 2006 the artampamp science of software testing u 21
w e b s e c u r i t y
John Weinschenk
President and Chief Executive Officer
gold sponsor
Donrsquot MissOut
On Another Issue of The
Test amp QAReport
e-newsletter
Each FREE weekly issue includes original articles that interview top thought leaders in software testingand quality trends best practices
and testQA methodologiesGet must-read articles that appear
only in this e-newsletter
Sign up atwwwstpmagcomtqa
Many tools exist that help QA departments and programmerstest applications However itrsquos also crucial that senior-
level software managers have the necessary information to inter-act with executive management and accurately assess the effec-tiveness of their development teams ldquoIf addressing software qual-ity early in the development cycle is an important priority for yourcompany Stelligent can helprdquo says Burke Cox Stelligentrsquos CEO
ldquoAs the thought leaders in early testing we use commercialand open-source technologies to assess and manage softwarequality during the development and assembly of applicationsrdquoCox says ldquoThe Quality Risk Index provides an objective meas-ure of software quality that enables an entire organization toassess quality and progressrdquo
Cox believes that Stelligent (wwwstelligentcom) transformsthe way software is developed and tested By introducing com-prehensive inspection as part of a continuous process both thedevelopment team and senior management gain crucial visibil-ity into the quality risks associated with software projects This
real-time feedback lets organ-izations manage quality longbefore products are everdelivered to the QA team forevaluation
Stelligentrsquos services suchas its Kickstart Quality RiskAssessment are used byorganizations developingsoftware using managedlanguages like C orJava providing bothstatic and dynamicanalysis on the appli-cation source andcompiled code
How effective isthe Kickstart Quality
Risk Assessment Wellone customer recentlyacquired a software
product that it believedwas of high quality based on a
due-diligence assessment of the source code andend user experience However when trying to integrate the prod-uct into its larger solution it found the product had poor toler-ance to change
Thatrsquos when the customer called Stelligent whose KickstartQuality Risk Assessment identified the problems Then its
Quality Roadmap and Seven-Point Implementation Plan identified top priorities for making programmatic changes through refactoring and introduced developer testing as an active part of the continuous integration process The customer
found that Stelligentrsquos QualityRoadmap quickly solved theproblem and then its Con-tinuous Quality program kept
them solved Case closedldquoFor many organizations our Quality Risk Index is the first
objective measure of their software quality that they have everseenrdquo Cox says ldquoThis helps the software manager by providingan independent assessment of what their quality practices cur-rently achieve When working with the executive managementteam in developing schedules approving budgets and other prod-uct management activities the software manager uses Stelligentreports as a basis for discussion For example high code com-plexity with poor test coverage might make a compelling argu-ment to delay release as opposed to simply stating that the prod-uct is just not ready for prime timerdquo
Stelligentrsquos services provide a critical and objective meas-urement of your softwarersquos quality Use this intelligence tomake it difficult to enter new defects into the source repos-itory helping your developers leverage the continuous qual-ity feedback before code even reaches the QA department
Stelligent is a subsidiary of JNetDirect which makesConvergence a solution for providing real-time visibility intosoftware quality CoView which helps your team develop accu-rate and effective JUnit tests and also high-performance JDBCdrivers But long before JNetDirect acquired Stelligent it was acustomer using the Kickstart Quality Risk Assessment servicesto help build its own products Stelligentrsquos impact was so dra-matic that JNetDirect immediately saw the value of bringingStelligentrsquos services to a broader audience
ldquoDefects will always enter the build systemrdquo Cox says ldquoOurbusiness is ensuring that they cannot live there for very longQuality may not have a material impact on initial product rev-enue but it is the most significant driver of product profitabil-ity Most companies struggle with describing quality it is mixedwith subjective measures and anecdotal evidence Our customershave a quality score they can point to when setting goals andbenchmarks For organizations interested in increasing earningslowering the costs associated with poor product quality is thebest place to startrdquo
Stelligentrsquos services start with the Kickstart Quality RiskAssessmentmdashask the company how its objective measures canhelp you too improve your software quality ampamp
Stelligent Brings ObjectivityTo Quality Measurement
t e s t i n g s e r v i c e s
Burke Cox
Chief Executive Officer
1 october 2006 the artampamp science of software testing u 23
silver sponsor
SPONSORED WHITE PAPER
SPONSORED WHITE PAPER
When it comes to software testing two key words are speedand accuracymdashnot only of the application yoursquore test-
ing but also of the QA process itself Thatrsquos what teams needmdashand what TechExcel delivers in DevTest one of the three toolsin its DevSuite DevTest offers your developers test standardi-zation test reuse and powerful defect analysis capabilities whilegiving managers a birdrsquos-eye view of the entire process
ldquoDevTest is an integrated solution that allows QA teams toimprove standardization and leverage existing testing knowl-edge to carefully monitor test executionrdquo says Tieren Zhou TechExcelrsquos CEO and chief software architect ldquoDevTest accomplish-es that goal by focusing knowledge management test librarycreation planning and scheduling and test execution and analy-sis I think our approach is truly unique to the marketrdquo
DevTest is a one-stop information resource for your teamstoring all related test documentation including requirementsdocuments specifications automation scripts screen shotsand other essential components in a central repository This
ldquoknowledge viewrdquo can beused in test creation andexecution so that the testteam is always equipped to plan and execute itsassignments
A test library is built onthis knowledge founda-tion letting your teamreuse tests for new testassignments new soft-ware versions or evenentirely different soft-ware products Thesestandard test pro-cedures called test templates can belinked to supporting
documentation in theknowledge view Testtemplates can be or-
ganized and classifiedbased on products applicable
environments functional areas or any otherstructure on which a team needs to focus
Once a test library has been created QA managers and teamleads use a wizard-driven interface to assign tasks to test teamshelping them leverage everything in the test library includingprevious test assignments and defect history to aid in the plan-
ning process The test team receives its test assignment in theDevTest interface executes the items assigned to it and sub-mits defects directly from the interface into an integrated defect-tracking tool Meanwhile DevTest tracks the test results in areal-time dashboard and in presentation-quality custom reports
In addition to DevTest TechExcel (wwwtechexcelcom) offerstwo other products in the DevSuite DevTrack and DevPlan
DevTrack tracks andmanages product defectschange requests and oth-er issues facilitating team-work among users teams
and customers DevTrack also provides workflow and processautomation robust searching and reporting and point-and-click customization
The newest member of the DevSuite DevPlan is an inno-vative project-tracking tool designed exclusively for applicationlife-cycle management DevPlan unites project tracking andissue management and incorporates configurable workflowsnotifications meeting requests and process automation
Zhou explains that in order to be effective a QA organiza-tion needs both preparation and education and that TechExcelrsquostools can help
ldquoDevTest provides all of the materials needed to create andexecute test assignments in one location so that leads and man-agers have the information they need to craft effective testsand testers have the information they need to execute themrdquohe says ldquoDevTestrsquos planning wizards are a favorite amongtest managers because they allow them to query existing datato help plan their assignments and reduce guesswork In addi-tion the built-in integration with DevTrack allows testers toexecute their test assignments and log and regress defects froma single interfacerdquo
In the near future the DevSuite will support distributeddeployment allowing global development organizations toachieve the benefits of local performance even when workingon a global projectmdashstay tuned for more about this excitingdevelopment
DevTest can help both small and large organizations get ahandle on quality assurance ldquoDevTest is useful in any develop-ment environmentrdquo Zhou says ldquoItrsquos well suited for large glob-al development organizations because it provides them with ascalable real-time view of their test projects regardless of whetherthe tests are executed by their core team by an outsourced teamor by auto-testing tools However even smaller developmentteams benefit from the ability to create manage analyze andreuse their test coveragerdquo ampamp
TechExcel Takes the Guesswork Out of Quality Assurance
a p p l i c a t i o n l i f e c y c l e m a n a g e m e n t
gold sponsor
Tieren Zhou
CEO and Chief Software Architect
1 october 2006 the artampamp science of software testing u 15
The Fast amp Scalable Team Solution forDefect amp Issue Tracking bull Feature amp Change Tracking bull Task amp To-do List Tracking bull Helpdesk Ticket Tracking
OnTime is the market-leading project defect and feature management tool for agile software development and test teamsOnTime facilitates tracking analyzing and trending team-based software development efforts in an intuitive and powerful user interface A fully customizable UI powerful workflow process enforcements two-way email communications and custom reports combine to help software development teams ship software on-time
Available for Windows Web amp VSNET 20032005
800middot653middot0024
wwwaxosoftcomso f tware fo r so f tware deve lopment trade
Ship Software OnTimetrade
Only $495 for up to 5 Users bull Only $995 for up to 10 UsersFree Single-User Installations
OnTime 2006 Small Team Edition
bull For Teams up to 10 Membersbull Free Single-User Installationsbull $495 for 5-Team Membersbull $995 for 10-Team Members
OnTime 2006 Professional Edition
bull For Teams up to 10 Membersbull Free Single-User Installationsbull $495 for 5-Team Membersbull $995 for 10-Team Members
bull For Teams of 10 to 1000 Membersbull From $149 Per User
SDTimesAd_OnTime2006indd 1 62706 14151 PM
Managing the entire scope of one or more software devel-opment projects can be daunting Within the sea of
tasks features defects and test cycles among team mem-bers managers stakeholders and customers just figuringout whorsquos working on what can be a challenge Thatrsquos where
Axosoftrsquos OnTime 2006enters the picture
OnTime 2006 is adefect management sys-tem designed to help soft-ware development teamsship software on timeWith a focus on ship-ping software on timeit not only effectivelytracks and managesbugs it effectivelyaddresses the broad-er challenges andbest practices ofproject manage-ment
OnTime 2006offers maximum
flexibility for adminis-trators and users alikewith ready access from
a Windows client a Webbrowser or within Visual Studio The highly
configurable and customizable application can also be inte-grated with leading SCM packages including Perforce SCMSourceGear Vault Subversion and Visual SourceSafe
In addition to tracking defects features and tasks thesoftware governs projects with highly customizable workflowand security rules It also offers e-mail notifications and con-versation thread tracking time tracking and work loggingcustom fields and reporting audit trails and archiving
While many of Axosoftrsquos clients switch to OnTime 2006from other products utilizing its importing functionalityothers seek to upgrade from a manual process
ldquoTraditionally development tasks requests defects andother items that occur over the course of the developmentcycle have been tracked in spreadsheets providing ampleopportunity for human error and miscommunicationbetween teamsrdquo says Dan Suceava chief software architectfor Axosoft (wwwaxosoftcom)
ldquoOnTime 2006 keeps track of everything and prevents
important items from slipping through the cracksrdquo heexplains ldquoInstead of wasting developer time with trivialupdates and report requests managers can easily pull theinformation themselves OnTime 2006 frees developers frommost if not all of the process-related overhead that sur-rounds a project and it helps them focus on what they dobest building great softwarerdquo
Developers who use OnTime 2006 see a clear intuitiveview of all of their projects including all the issues defectsfeature requests milestones and tasks As they completetasks developers escalate those items to subsequent work-flow steps This can be set to automatically e-mail otherteam members when the next set of actions is ready to beperformed
OnTime 2006 allows project managers to define speci-fications workflows and security rules They can create ldquowhatifrdquo scenarios that generate predictions for completion timesworkload distribution and other milestones During the pro-jectrsquos execution project managers have access to total proj-ect visibility They know who is working on what the progressbeing made on tasks where bottlenecks are occurring defectrates and estimated completion times
After a project has been completed managers can takeadvantage of the accurate project history that continues toreside within OnTime 2006 This information can be usedfor anything from measuring various productivity rates toproviding a basis for decisions concerning future projectsThus past projects become part of a living knowledge basethat can be consulted at any time
Axosoft even offers a Customer Portal add-on for OnTime2006 that embraces customers as participants in the processof shipping software on time It provides a Web interfacetied to the OnTime database where customers can submit
bugs and other inputCustomizable security set-tings determine how muchproject information will bevisible to customers This
functionality is especially useful for consultants and ISVsduring beta-testing phases
More functionality is in the works for the productldquoOnTime 2006 is already a tool designed to enable the entiredevelopment team to ship software on timerdquo explainsSuceava ldquoWhile today it meets the needs of project man-agers developers and testers extremely well future versionswill provide further functionality for support professionalsIT directors and executivesrdquo ampamp
Axosoft Keeps DevelopmentProjects on the Fast Track
1 october 2006 the artampamp science of software testing u 17
d e f e c t t r a c k i n g
gold sponsor
Dan Suceava
Chief Software Architect
Software configuration management isnrsquot a luxurymdashforany modern software development team itrsquos a necessi-
ty But when an SCM system is as easy to use easy to admin-ister and full of productivity-enhancing features and bene-fits as Perforcersquos system itrsquoll feel like all of your developersare traveling in first class all the way
ldquoThe Perforce SCM System lets teams of local and dis-tributed developers share project files that are centrally storedand managed by the Perforce Serverrdquo says John WalkerPerforcersquos principal product consultant ldquoThe Perforce Serverhandles user requests and tracks all development activity inthe built-in Perforce database Each filersquos state informationcan be quickly discerned from any of Perforcersquos cross-platform clients This means that a user can see when a ver-sion of a particular file has been updated deleted or addedto the server Icons associated with the files indicate whethertheyrsquore currently being edited added or deleted by otherdevelopersrdquo
The Perforce SCM System(wwwperforcecom) letseach developer on your teamobtain and refresh a localprivate copy of versionedfiles by synchronizing themwith files stored in a filedepot Since all of themetadata is centralizedfile state information canbe gathered quickly andeasily Perforce efficient-ly manages both bina-ry and text files fromsource code to graph-ics to documentationThatrsquos one reasonwhy Perforce has
been adopted not onlyby pure developmentshops but also by
chip and hardwaremanufacturers that maintain
large binary assets For example Walker says the Perforce System is very pop-
ular in the game development industry where artists arerequired to create and store large numbers of large imagefiles To help all usersmdashnot just software developersmdashthePerforce software automatically renders thumbnail versions
of image files that can be viewed from within the applica-tionrsquos cross-platform graphical client P4V Regardless ofwhether yoursquore working with text or binary artifacts Perforcehandles the job with style
Perforcersquos clientserverapplication operates overany network or the In-ternet and includes itsown internal database
enginemdashsaving you money improving performance and get-ting the system up and running fast Installing the PerforceServer is simple Download the installation set from the com-panyrsquos Web site and run it Installing any one of a varietyof Perforce clients is also straightforward and the resourcesneeded to manage the installation are minimal Even largedistributed sites of 2000 or more users can be maintainedby a single administrator explains Walker
While Perforce has the basics down pat the real payoff isin the extras that the softwarersquos creators developed to makeyour own coders more efficient Perforcersquos intelligent branch-ing mechanism lets developers work on different releasebranches of a particular application in parallel Code linessupporting specific releases are clearly and visually definedin the system Once a branch is created the relationship itshares with the parent branch is tracked in the serverrsquos meta-data Since the ancestry of files is tracked between branch-es the integration history is maintained in the metadataThat makes it easy to see which changes have and havenrsquotbeen integrated between the related branchesmdashwhether yoursquoreworking in Java J2EE NET CC++ C Visual Basic orHTML
Perforce provides a basic defect-tracking system called jobsA job typically represents an enhancement request or a bugto be fixed Job definitions are customizable to support work-flow and jobs can also work with leading third-party defecttracking systems Support for Mercuryrsquos Quality Center isplanned for the next release This integration will allow usersto enter bugs in Quality Center and have them replicatedinto Perforce as jobs
ldquoThe Perforce SCM Systemrsquos high performance is the prod-uct of a streamlined architecture and closely integrated imple-mentationmdashnot expensive server or network hardwarerdquoWalker says ldquoThe Perforce Server does not require dedicat-ed hardware and client workstations never need upgradingThe systemrsquos networking capabilities arenrsquot a significant loadfor a typical LAN With little need for customization or con-figuration you can be up and running in minutesrdquo ampamp
Perforce Makes SCM FastEasy and First-Class
s o f t w a r e c o n f i g u r a t i o n m a n a g e m e n t
gold sponsor
John Walker
Principal Product Consultant
1 october 2006 the artampamp science of software testing u 19
The faster you find a security problem in the application lifecycle the better If developers catch vulnerabilities early they
can be fixed prior to pushing apps off to QA and customersFixing problems after they slip into a product can cost far morethan catching them early And of course if you donrsquot find the
defects someone else willldquoCenzic solutions provide
tremendous efficiencies andan immediate return on investmentrdquo says JohnWeinschenk Cenzicrsquospresident and CEO (wwwcenziccom) ldquoBy auto-mating the security test-ing process our cus-tomers can secure theirapplications faster andless expensively Likeperformance andfunctionality testingsecurity testingshould be automat-ed Vulnerabilitiesare an open invita-
tion to hackers AllWeb applications andinfrastructure can be
tested with Cenzic solutionsto locate security problemsrdquo
Cenzicrsquos two key security products are Cenzic Hailstorm andCenzic ClickToSecure These solutions provide the most com-prehensive and accurate tests in the industry Weinschenk saysthat while most vendors use scanning technology to focus onlyon commonly known vulnerabilities Hailstorm uses Cenzicrsquospatent-pending Stateful Assessment technology to automatepenetration testing by using a series of transactions to identifyvulnerabilities
Plus Hailstormrsquos SmartAttack Objects Library takes thesoftware beyond merely finding security holesmdashit can also helpenforce internal policies as well as bring organizations intoregulatory compliance with rules like the Gramm-Leach-BlileyAct (GLBA) SB1386 and Sarbanes-Oxley and the PaymentCard Industry (PCI) Data Security Standard and with bestpractices including SANS and OWASP in addition to manyother regulatory standards
ClickToSecure is a service that allows developers to make useof Software as a Service (SaaS) Tests are conducted remotely by
Cenzicrsquos own security expertsmdashand that means that your devel-opment team can build secure software even if it doesnrsquot haveexpertise in that area ClickToSecure accesses applicationsusing a combination of Hailstorm technology and the CenzicIntelligent Analysis (CIA) Research Lab to run assessments
In fact that CIA Lab is the foundation of the SmartAttackObjects Library Cenzic provides continuous updates based onnew vulnerabilities through the lab similar to an anti-virusmodel to help you stay ahead of the exploits and attacks Youcan use the library just as it is plus you can use CenzicrsquosSmartAttack Objects as templates that your own security archi-tects can customize for your organizationrsquos special require-ments if necessary
In addition to its software applications and services Cenzicoffers security training courses that give customers the knowl-edge and skills to use and maintain its products successfullyThe company also offers consulting services that include on-site assessment methodology and software-engineering andconsulting implementation with professional vulnerabilityconsultants who have experience in penetration testing andethical hacking
Among Cenzicrsquos many customers are Boston College DebtExchange IRIS Link and K2 Networks Boston Collegersquos devel-opment group uses Hailstorm in-house throughout its infor-mation security group to test all university Web applicationsAs they find security weaknesses developers use Hailstormreports to remediate those vulnerabilities The same group alsoaddresses its regulatory compliance issues with Hailstorm
In the case of Debt Exchange IRIS Link and K2 NetworksClickToSecure is put to work to perform security assessmentsCustomers call or fax requests to test their applications andCenzic experts test them remotely using Hailstorm in collabo-ration with the CIA Labrsquos expertise Detailed results are then
presented to the customeralong with detailed remedi-ation information
Weinschenk says thatCenzic solutions are a perfect fit for development managersand other executives Using a dashboard customers can viewapplications in the testing phase as well as the number andtypes of vulnerabilities that are found
Weinschenk adds that testing is extremely importantbecause susceptibility to intrusion can result in major recoverycosts and regulatory penalties He also reports that Cenzic hasnt tested a single application that was not vulnerable Butdonrsquot worrymdashCenzic finds the vulnerabilities before the badguys do Put Cenzic to work for you ampamp
Keeping the Bad Guys At Bay With Cenzic Solutions
1 october 2006 the artampamp science of software testing u 21
w e b s e c u r i t y
John Weinschenk
President and Chief Executive Officer
gold sponsor
Donrsquot MissOut
On Another Issue of The
Test amp QAReport
e-newsletter
Each FREE weekly issue includes original articles that interview top thought leaders in software testingand quality trends best practices
and testQA methodologiesGet must-read articles that appear
only in this e-newsletter
Sign up atwwwstpmagcomtqa
Many tools exist that help QA departments and programmerstest applications However itrsquos also crucial that senior-
level software managers have the necessary information to inter-act with executive management and accurately assess the effec-tiveness of their development teams ldquoIf addressing software qual-ity early in the development cycle is an important priority for yourcompany Stelligent can helprdquo says Burke Cox Stelligentrsquos CEO
ldquoAs the thought leaders in early testing we use commercialand open-source technologies to assess and manage softwarequality during the development and assembly of applicationsrdquoCox says ldquoThe Quality Risk Index provides an objective meas-ure of software quality that enables an entire organization toassess quality and progressrdquo
Cox believes that Stelligent (wwwstelligentcom) transformsthe way software is developed and tested By introducing com-prehensive inspection as part of a continuous process both thedevelopment team and senior management gain crucial visibil-ity into the quality risks associated with software projects This
real-time feedback lets organ-izations manage quality longbefore products are everdelivered to the QA team forevaluation
Stelligentrsquos services suchas its Kickstart Quality RiskAssessment are used byorganizations developingsoftware using managedlanguages like C orJava providing bothstatic and dynamicanalysis on the appli-cation source andcompiled code
How effective isthe Kickstart Quality
Risk Assessment Wellone customer recentlyacquired a software
product that it believedwas of high quality based on a
due-diligence assessment of the source code andend user experience However when trying to integrate the prod-uct into its larger solution it found the product had poor toler-ance to change
Thatrsquos when the customer called Stelligent whose KickstartQuality Risk Assessment identified the problems Then its
Quality Roadmap and Seven-Point Implementation Plan identified top priorities for making programmatic changes through refactoring and introduced developer testing as an active part of the continuous integration process The customer
found that Stelligentrsquos QualityRoadmap quickly solved theproblem and then its Con-tinuous Quality program kept
them solved Case closedldquoFor many organizations our Quality Risk Index is the first
objective measure of their software quality that they have everseenrdquo Cox says ldquoThis helps the software manager by providingan independent assessment of what their quality practices cur-rently achieve When working with the executive managementteam in developing schedules approving budgets and other prod-uct management activities the software manager uses Stelligentreports as a basis for discussion For example high code com-plexity with poor test coverage might make a compelling argu-ment to delay release as opposed to simply stating that the prod-uct is just not ready for prime timerdquo
Stelligentrsquos services provide a critical and objective meas-urement of your softwarersquos quality Use this intelligence tomake it difficult to enter new defects into the source repos-itory helping your developers leverage the continuous qual-ity feedback before code even reaches the QA department
Stelligent is a subsidiary of JNetDirect which makesConvergence a solution for providing real-time visibility intosoftware quality CoView which helps your team develop accu-rate and effective JUnit tests and also high-performance JDBCdrivers But long before JNetDirect acquired Stelligent it was acustomer using the Kickstart Quality Risk Assessment servicesto help build its own products Stelligentrsquos impact was so dra-matic that JNetDirect immediately saw the value of bringingStelligentrsquos services to a broader audience
ldquoDefects will always enter the build systemrdquo Cox says ldquoOurbusiness is ensuring that they cannot live there for very longQuality may not have a material impact on initial product rev-enue but it is the most significant driver of product profitabil-ity Most companies struggle with describing quality it is mixedwith subjective measures and anecdotal evidence Our customershave a quality score they can point to when setting goals andbenchmarks For organizations interested in increasing earningslowering the costs associated with poor product quality is thebest place to startrdquo
Stelligentrsquos services start with the Kickstart Quality RiskAssessmentmdashask the company how its objective measures canhelp you too improve your software quality ampamp
Stelligent Brings ObjectivityTo Quality Measurement
t e s t i n g s e r v i c e s
Burke Cox
Chief Executive Officer
1 october 2006 the artampamp science of software testing u 23
silver sponsor
SPONSORED WHITE PAPER
When it comes to software testing two key words are speedand accuracymdashnot only of the application yoursquore test-
ing but also of the QA process itself Thatrsquos what teams needmdashand what TechExcel delivers in DevTest one of the three toolsin its DevSuite DevTest offers your developers test standardi-zation test reuse and powerful defect analysis capabilities whilegiving managers a birdrsquos-eye view of the entire process
ldquoDevTest is an integrated solution that allows QA teams toimprove standardization and leverage existing testing knowl-edge to carefully monitor test executionrdquo says Tieren Zhou TechExcelrsquos CEO and chief software architect ldquoDevTest accomplish-es that goal by focusing knowledge management test librarycreation planning and scheduling and test execution and analy-sis I think our approach is truly unique to the marketrdquo
DevTest is a one-stop information resource for your teamstoring all related test documentation including requirementsdocuments specifications automation scripts screen shotsand other essential components in a central repository This
ldquoknowledge viewrdquo can beused in test creation andexecution so that the testteam is always equipped to plan and execute itsassignments
A test library is built onthis knowledge founda-tion letting your teamreuse tests for new testassignments new soft-ware versions or evenentirely different soft-ware products Thesestandard test pro-cedures called test templates can belinked to supporting
documentation in theknowledge view Testtemplates can be or-
ganized and classifiedbased on products applicable
environments functional areas or any otherstructure on which a team needs to focus
Once a test library has been created QA managers and teamleads use a wizard-driven interface to assign tasks to test teamshelping them leverage everything in the test library includingprevious test assignments and defect history to aid in the plan-
ning process The test team receives its test assignment in theDevTest interface executes the items assigned to it and sub-mits defects directly from the interface into an integrated defect-tracking tool Meanwhile DevTest tracks the test results in areal-time dashboard and in presentation-quality custom reports
In addition to DevTest TechExcel (wwwtechexcelcom) offerstwo other products in the DevSuite DevTrack and DevPlan
DevTrack tracks andmanages product defectschange requests and oth-er issues facilitating team-work among users teams
and customers DevTrack also provides workflow and processautomation robust searching and reporting and point-and-click customization
The newest member of the DevSuite DevPlan is an inno-vative project-tracking tool designed exclusively for applicationlife-cycle management DevPlan unites project tracking andissue management and incorporates configurable workflowsnotifications meeting requests and process automation
Zhou explains that in order to be effective a QA organiza-tion needs both preparation and education and that TechExcelrsquostools can help
ldquoDevTest provides all of the materials needed to create andexecute test assignments in one location so that leads and man-agers have the information they need to craft effective testsand testers have the information they need to execute themrdquohe says ldquoDevTestrsquos planning wizards are a favorite amongtest managers because they allow them to query existing datato help plan their assignments and reduce guesswork In addi-tion the built-in integration with DevTrack allows testers toexecute their test assignments and log and regress defects froma single interfacerdquo
In the near future the DevSuite will support distributeddeployment allowing global development organizations toachieve the benefits of local performance even when workingon a global projectmdashstay tuned for more about this excitingdevelopment
DevTest can help both small and large organizations get ahandle on quality assurance ldquoDevTest is useful in any develop-ment environmentrdquo Zhou says ldquoItrsquos well suited for large glob-al development organizations because it provides them with ascalable real-time view of their test projects regardless of whetherthe tests are executed by their core team by an outsourced teamor by auto-testing tools However even smaller developmentteams benefit from the ability to create manage analyze andreuse their test coveragerdquo ampamp
TechExcel Takes the Guesswork Out of Quality Assurance
a p p l i c a t i o n l i f e c y c l e m a n a g e m e n t
gold sponsor
Tieren Zhou
CEO and Chief Software Architect
1 october 2006 the artampamp science of software testing u 15
The Fast amp Scalable Team Solution forDefect amp Issue Tracking bull Feature amp Change Tracking bull Task amp To-do List Tracking bull Helpdesk Ticket Tracking
OnTime is the market-leading project defect and feature management tool for agile software development and test teamsOnTime facilitates tracking analyzing and trending team-based software development efforts in an intuitive and powerful user interface A fully customizable UI powerful workflow process enforcements two-way email communications and custom reports combine to help software development teams ship software on-time
Available for Windows Web amp VSNET 20032005
800middot653middot0024
wwwaxosoftcomso f tware fo r so f tware deve lopment trade
Ship Software OnTimetrade
Only $495 for up to 5 Users bull Only $995 for up to 10 UsersFree Single-User Installations
OnTime 2006 Small Team Edition
bull For Teams up to 10 Membersbull Free Single-User Installationsbull $495 for 5-Team Membersbull $995 for 10-Team Members
OnTime 2006 Professional Edition
bull For Teams up to 10 Membersbull Free Single-User Installationsbull $495 for 5-Team Membersbull $995 for 10-Team Members
bull For Teams of 10 to 1000 Membersbull From $149 Per User
SDTimesAd_OnTime2006indd 1 62706 14151 PM
Managing the entire scope of one or more software devel-opment projects can be daunting Within the sea of
tasks features defects and test cycles among team mem-bers managers stakeholders and customers just figuringout whorsquos working on what can be a challenge Thatrsquos where
Axosoftrsquos OnTime 2006enters the picture
OnTime 2006 is adefect management sys-tem designed to help soft-ware development teamsship software on timeWith a focus on ship-ping software on timeit not only effectivelytracks and managesbugs it effectivelyaddresses the broad-er challenges andbest practices ofproject manage-ment
OnTime 2006offers maximum
flexibility for adminis-trators and users alikewith ready access from
a Windows client a Webbrowser or within Visual Studio The highly
configurable and customizable application can also be inte-grated with leading SCM packages including Perforce SCMSourceGear Vault Subversion and Visual SourceSafe
In addition to tracking defects features and tasks thesoftware governs projects with highly customizable workflowand security rules It also offers e-mail notifications and con-versation thread tracking time tracking and work loggingcustom fields and reporting audit trails and archiving
While many of Axosoftrsquos clients switch to OnTime 2006from other products utilizing its importing functionalityothers seek to upgrade from a manual process
ldquoTraditionally development tasks requests defects andother items that occur over the course of the developmentcycle have been tracked in spreadsheets providing ampleopportunity for human error and miscommunicationbetween teamsrdquo says Dan Suceava chief software architectfor Axosoft (wwwaxosoftcom)
ldquoOnTime 2006 keeps track of everything and prevents
important items from slipping through the cracksrdquo heexplains ldquoInstead of wasting developer time with trivialupdates and report requests managers can easily pull theinformation themselves OnTime 2006 frees developers frommost if not all of the process-related overhead that sur-rounds a project and it helps them focus on what they dobest building great softwarerdquo
Developers who use OnTime 2006 see a clear intuitiveview of all of their projects including all the issues defectsfeature requests milestones and tasks As they completetasks developers escalate those items to subsequent work-flow steps This can be set to automatically e-mail otherteam members when the next set of actions is ready to beperformed
OnTime 2006 allows project managers to define speci-fications workflows and security rules They can create ldquowhatifrdquo scenarios that generate predictions for completion timesworkload distribution and other milestones During the pro-jectrsquos execution project managers have access to total proj-ect visibility They know who is working on what the progressbeing made on tasks where bottlenecks are occurring defectrates and estimated completion times
After a project has been completed managers can takeadvantage of the accurate project history that continues toreside within OnTime 2006 This information can be usedfor anything from measuring various productivity rates toproviding a basis for decisions concerning future projectsThus past projects become part of a living knowledge basethat can be consulted at any time
Axosoft even offers a Customer Portal add-on for OnTime2006 that embraces customers as participants in the processof shipping software on time It provides a Web interfacetied to the OnTime database where customers can submit
bugs and other inputCustomizable security set-tings determine how muchproject information will bevisible to customers This
functionality is especially useful for consultants and ISVsduring beta-testing phases
More functionality is in the works for the productldquoOnTime 2006 is already a tool designed to enable the entiredevelopment team to ship software on timerdquo explainsSuceava ldquoWhile today it meets the needs of project man-agers developers and testers extremely well future versionswill provide further functionality for support professionalsIT directors and executivesrdquo ampamp
Axosoft Keeps DevelopmentProjects on the Fast Track
1 october 2006 the artampamp science of software testing u 17
d e f e c t t r a c k i n g
gold sponsor
Dan Suceava
Chief Software Architect
Software configuration management isnrsquot a luxurymdashforany modern software development team itrsquos a necessi-
ty But when an SCM system is as easy to use easy to admin-ister and full of productivity-enhancing features and bene-fits as Perforcersquos system itrsquoll feel like all of your developersare traveling in first class all the way
ldquoThe Perforce SCM System lets teams of local and dis-tributed developers share project files that are centrally storedand managed by the Perforce Serverrdquo says John WalkerPerforcersquos principal product consultant ldquoThe Perforce Serverhandles user requests and tracks all development activity inthe built-in Perforce database Each filersquos state informationcan be quickly discerned from any of Perforcersquos cross-platform clients This means that a user can see when a ver-sion of a particular file has been updated deleted or addedto the server Icons associated with the files indicate whethertheyrsquore currently being edited added or deleted by otherdevelopersrdquo
The Perforce SCM System(wwwperforcecom) letseach developer on your teamobtain and refresh a localprivate copy of versionedfiles by synchronizing themwith files stored in a filedepot Since all of themetadata is centralizedfile state information canbe gathered quickly andeasily Perforce efficient-ly manages both bina-ry and text files fromsource code to graph-ics to documentationThatrsquos one reasonwhy Perforce has
been adopted not onlyby pure developmentshops but also by
chip and hardwaremanufacturers that maintain
large binary assets For example Walker says the Perforce System is very pop-
ular in the game development industry where artists arerequired to create and store large numbers of large imagefiles To help all usersmdashnot just software developersmdashthePerforce software automatically renders thumbnail versions
of image files that can be viewed from within the applica-tionrsquos cross-platform graphical client P4V Regardless ofwhether yoursquore working with text or binary artifacts Perforcehandles the job with style
Perforcersquos clientserverapplication operates overany network or the In-ternet and includes itsown internal database
enginemdashsaving you money improving performance and get-ting the system up and running fast Installing the PerforceServer is simple Download the installation set from the com-panyrsquos Web site and run it Installing any one of a varietyof Perforce clients is also straightforward and the resourcesneeded to manage the installation are minimal Even largedistributed sites of 2000 or more users can be maintainedby a single administrator explains Walker
While Perforce has the basics down pat the real payoff isin the extras that the softwarersquos creators developed to makeyour own coders more efficient Perforcersquos intelligent branch-ing mechanism lets developers work on different releasebranches of a particular application in parallel Code linessupporting specific releases are clearly and visually definedin the system Once a branch is created the relationship itshares with the parent branch is tracked in the serverrsquos meta-data Since the ancestry of files is tracked between branch-es the integration history is maintained in the metadataThat makes it easy to see which changes have and havenrsquotbeen integrated between the related branchesmdashwhether yoursquoreworking in Java J2EE NET CC++ C Visual Basic orHTML
Perforce provides a basic defect-tracking system called jobsA job typically represents an enhancement request or a bugto be fixed Job definitions are customizable to support work-flow and jobs can also work with leading third-party defecttracking systems Support for Mercuryrsquos Quality Center isplanned for the next release This integration will allow usersto enter bugs in Quality Center and have them replicatedinto Perforce as jobs
ldquoThe Perforce SCM Systemrsquos high performance is the prod-uct of a streamlined architecture and closely integrated imple-mentationmdashnot expensive server or network hardwarerdquoWalker says ldquoThe Perforce Server does not require dedicat-ed hardware and client workstations never need upgradingThe systemrsquos networking capabilities arenrsquot a significant loadfor a typical LAN With little need for customization or con-figuration you can be up and running in minutesrdquo ampamp
Perforce Makes SCM FastEasy and First-Class
s o f t w a r e c o n f i g u r a t i o n m a n a g e m e n t
gold sponsor
John Walker
Principal Product Consultant
1 october 2006 the artampamp science of software testing u 19
The faster you find a security problem in the application lifecycle the better If developers catch vulnerabilities early they
can be fixed prior to pushing apps off to QA and customersFixing problems after they slip into a product can cost far morethan catching them early And of course if you donrsquot find the
defects someone else willldquoCenzic solutions provide
tremendous efficiencies andan immediate return on investmentrdquo says JohnWeinschenk Cenzicrsquospresident and CEO (wwwcenziccom) ldquoBy auto-mating the security test-ing process our cus-tomers can secure theirapplications faster andless expensively Likeperformance andfunctionality testingsecurity testingshould be automat-ed Vulnerabilitiesare an open invita-
tion to hackers AllWeb applications andinfrastructure can be
tested with Cenzic solutionsto locate security problemsrdquo
Cenzicrsquos two key security products are Cenzic Hailstorm andCenzic ClickToSecure These solutions provide the most com-prehensive and accurate tests in the industry Weinschenk saysthat while most vendors use scanning technology to focus onlyon commonly known vulnerabilities Hailstorm uses Cenzicrsquospatent-pending Stateful Assessment technology to automatepenetration testing by using a series of transactions to identifyvulnerabilities
Plus Hailstormrsquos SmartAttack Objects Library takes thesoftware beyond merely finding security holesmdashit can also helpenforce internal policies as well as bring organizations intoregulatory compliance with rules like the Gramm-Leach-BlileyAct (GLBA) SB1386 and Sarbanes-Oxley and the PaymentCard Industry (PCI) Data Security Standard and with bestpractices including SANS and OWASP in addition to manyother regulatory standards
ClickToSecure is a service that allows developers to make useof Software as a Service (SaaS) Tests are conducted remotely by
Cenzicrsquos own security expertsmdashand that means that your devel-opment team can build secure software even if it doesnrsquot haveexpertise in that area ClickToSecure accesses applicationsusing a combination of Hailstorm technology and the CenzicIntelligent Analysis (CIA) Research Lab to run assessments
In fact that CIA Lab is the foundation of the SmartAttackObjects Library Cenzic provides continuous updates based onnew vulnerabilities through the lab similar to an anti-virusmodel to help you stay ahead of the exploits and attacks Youcan use the library just as it is plus you can use CenzicrsquosSmartAttack Objects as templates that your own security archi-tects can customize for your organizationrsquos special require-ments if necessary
In addition to its software applications and services Cenzicoffers security training courses that give customers the knowl-edge and skills to use and maintain its products successfullyThe company also offers consulting services that include on-site assessment methodology and software-engineering andconsulting implementation with professional vulnerabilityconsultants who have experience in penetration testing andethical hacking
Among Cenzicrsquos many customers are Boston College DebtExchange IRIS Link and K2 Networks Boston Collegersquos devel-opment group uses Hailstorm in-house throughout its infor-mation security group to test all university Web applicationsAs they find security weaknesses developers use Hailstormreports to remediate those vulnerabilities The same group alsoaddresses its regulatory compliance issues with Hailstorm
In the case of Debt Exchange IRIS Link and K2 NetworksClickToSecure is put to work to perform security assessmentsCustomers call or fax requests to test their applications andCenzic experts test them remotely using Hailstorm in collabo-ration with the CIA Labrsquos expertise Detailed results are then
presented to the customeralong with detailed remedi-ation information
Weinschenk says thatCenzic solutions are a perfect fit for development managersand other executives Using a dashboard customers can viewapplications in the testing phase as well as the number andtypes of vulnerabilities that are found
Weinschenk adds that testing is extremely importantbecause susceptibility to intrusion can result in major recoverycosts and regulatory penalties He also reports that Cenzic hasnt tested a single application that was not vulnerable Butdonrsquot worrymdashCenzic finds the vulnerabilities before the badguys do Put Cenzic to work for you ampamp
Keeping the Bad Guys At Bay With Cenzic Solutions
1 october 2006 the artampamp science of software testing u 21
w e b s e c u r i t y
John Weinschenk
President and Chief Executive Officer
gold sponsor
Donrsquot MissOut
On Another Issue of The
Test amp QAReport
e-newsletter
Each FREE weekly issue includes original articles that interview top thought leaders in software testingand quality trends best practices
and testQA methodologiesGet must-read articles that appear
only in this e-newsletter
Sign up atwwwstpmagcomtqa
Many tools exist that help QA departments and programmerstest applications However itrsquos also crucial that senior-
level software managers have the necessary information to inter-act with executive management and accurately assess the effec-tiveness of their development teams ldquoIf addressing software qual-ity early in the development cycle is an important priority for yourcompany Stelligent can helprdquo says Burke Cox Stelligentrsquos CEO
ldquoAs the thought leaders in early testing we use commercialand open-source technologies to assess and manage softwarequality during the development and assembly of applicationsrdquoCox says ldquoThe Quality Risk Index provides an objective meas-ure of software quality that enables an entire organization toassess quality and progressrdquo
Cox believes that Stelligent (wwwstelligentcom) transformsthe way software is developed and tested By introducing com-prehensive inspection as part of a continuous process both thedevelopment team and senior management gain crucial visibil-ity into the quality risks associated with software projects This
real-time feedback lets organ-izations manage quality longbefore products are everdelivered to the QA team forevaluation
Stelligentrsquos services suchas its Kickstart Quality RiskAssessment are used byorganizations developingsoftware using managedlanguages like C orJava providing bothstatic and dynamicanalysis on the appli-cation source andcompiled code
How effective isthe Kickstart Quality
Risk Assessment Wellone customer recentlyacquired a software
product that it believedwas of high quality based on a
due-diligence assessment of the source code andend user experience However when trying to integrate the prod-uct into its larger solution it found the product had poor toler-ance to change
Thatrsquos when the customer called Stelligent whose KickstartQuality Risk Assessment identified the problems Then its
Quality Roadmap and Seven-Point Implementation Plan identified top priorities for making programmatic changes through refactoring and introduced developer testing as an active part of the continuous integration process The customer
found that Stelligentrsquos QualityRoadmap quickly solved theproblem and then its Con-tinuous Quality program kept
them solved Case closedldquoFor many organizations our Quality Risk Index is the first
objective measure of their software quality that they have everseenrdquo Cox says ldquoThis helps the software manager by providingan independent assessment of what their quality practices cur-rently achieve When working with the executive managementteam in developing schedules approving budgets and other prod-uct management activities the software manager uses Stelligentreports as a basis for discussion For example high code com-plexity with poor test coverage might make a compelling argu-ment to delay release as opposed to simply stating that the prod-uct is just not ready for prime timerdquo
Stelligentrsquos services provide a critical and objective meas-urement of your softwarersquos quality Use this intelligence tomake it difficult to enter new defects into the source repos-itory helping your developers leverage the continuous qual-ity feedback before code even reaches the QA department
Stelligent is a subsidiary of JNetDirect which makesConvergence a solution for providing real-time visibility intosoftware quality CoView which helps your team develop accu-rate and effective JUnit tests and also high-performance JDBCdrivers But long before JNetDirect acquired Stelligent it was acustomer using the Kickstart Quality Risk Assessment servicesto help build its own products Stelligentrsquos impact was so dra-matic that JNetDirect immediately saw the value of bringingStelligentrsquos services to a broader audience
ldquoDefects will always enter the build systemrdquo Cox says ldquoOurbusiness is ensuring that they cannot live there for very longQuality may not have a material impact on initial product rev-enue but it is the most significant driver of product profitabil-ity Most companies struggle with describing quality it is mixedwith subjective measures and anecdotal evidence Our customershave a quality score they can point to when setting goals andbenchmarks For organizations interested in increasing earningslowering the costs associated with poor product quality is thebest place to startrdquo
Stelligentrsquos services start with the Kickstart Quality RiskAssessmentmdashask the company how its objective measures canhelp you too improve your software quality ampamp
Stelligent Brings ObjectivityTo Quality Measurement
t e s t i n g s e r v i c e s
Burke Cox
Chief Executive Officer
1 october 2006 the artampamp science of software testing u 23
silver sponsor
When it comes to software testing two key words are speedand accuracymdashnot only of the application yoursquore test-
ing but also of the QA process itself Thatrsquos what teams needmdashand what TechExcel delivers in DevTest one of the three toolsin its DevSuite DevTest offers your developers test standardi-zation test reuse and powerful defect analysis capabilities whilegiving managers a birdrsquos-eye view of the entire process
ldquoDevTest is an integrated solution that allows QA teams toimprove standardization and leverage existing testing knowl-edge to carefully monitor test executionrdquo says Tieren Zhou TechExcelrsquos CEO and chief software architect ldquoDevTest accomplish-es that goal by focusing knowledge management test librarycreation planning and scheduling and test execution and analy-sis I think our approach is truly unique to the marketrdquo
DevTest is a one-stop information resource for your teamstoring all related test documentation including requirementsdocuments specifications automation scripts screen shotsand other essential components in a central repository This
ldquoknowledge viewrdquo can beused in test creation andexecution so that the testteam is always equipped to plan and execute itsassignments
A test library is built onthis knowledge founda-tion letting your teamreuse tests for new testassignments new soft-ware versions or evenentirely different soft-ware products Thesestandard test pro-cedures called test templates can belinked to supporting
documentation in theknowledge view Testtemplates can be or-
ganized and classifiedbased on products applicable
environments functional areas or any otherstructure on which a team needs to focus
Once a test library has been created QA managers and teamleads use a wizard-driven interface to assign tasks to test teamshelping them leverage everything in the test library includingprevious test assignments and defect history to aid in the plan-
ning process The test team receives its test assignment in theDevTest interface executes the items assigned to it and sub-mits defects directly from the interface into an integrated defect-tracking tool Meanwhile DevTest tracks the test results in areal-time dashboard and in presentation-quality custom reports
In addition to DevTest TechExcel (wwwtechexcelcom) offerstwo other products in the DevSuite DevTrack and DevPlan
DevTrack tracks andmanages product defectschange requests and oth-er issues facilitating team-work among users teams
and customers DevTrack also provides workflow and processautomation robust searching and reporting and point-and-click customization
The newest member of the DevSuite DevPlan is an inno-vative project-tracking tool designed exclusively for applicationlife-cycle management DevPlan unites project tracking andissue management and incorporates configurable workflowsnotifications meeting requests and process automation
Zhou explains that in order to be effective a QA organiza-tion needs both preparation and education and that TechExcelrsquostools can help
ldquoDevTest provides all of the materials needed to create andexecute test assignments in one location so that leads and man-agers have the information they need to craft effective testsand testers have the information they need to execute themrdquohe says ldquoDevTestrsquos planning wizards are a favorite amongtest managers because they allow them to query existing datato help plan their assignments and reduce guesswork In addi-tion the built-in integration with DevTrack allows testers toexecute their test assignments and log and regress defects froma single interfacerdquo
In the near future the DevSuite will support distributeddeployment allowing global development organizations toachieve the benefits of local performance even when workingon a global projectmdashstay tuned for more about this excitingdevelopment
DevTest can help both small and large organizations get ahandle on quality assurance ldquoDevTest is useful in any develop-ment environmentrdquo Zhou says ldquoItrsquos well suited for large glob-al development organizations because it provides them with ascalable real-time view of their test projects regardless of whetherthe tests are executed by their core team by an outsourced teamor by auto-testing tools However even smaller developmentteams benefit from the ability to create manage analyze andreuse their test coveragerdquo ampamp
TechExcel Takes the Guesswork Out of Quality Assurance
a p p l i c a t i o n l i f e c y c l e m a n a g e m e n t
gold sponsor
Tieren Zhou
CEO and Chief Software Architect
1 october 2006 the artampamp science of software testing u 15
The Fast amp Scalable Team Solution forDefect amp Issue Tracking bull Feature amp Change Tracking bull Task amp To-do List Tracking bull Helpdesk Ticket Tracking
OnTime is the market-leading project defect and feature management tool for agile software development and test teamsOnTime facilitates tracking analyzing and trending team-based software development efforts in an intuitive and powerful user interface A fully customizable UI powerful workflow process enforcements two-way email communications and custom reports combine to help software development teams ship software on-time
Available for Windows Web amp VSNET 20032005
800middot653middot0024
wwwaxosoftcomso f tware fo r so f tware deve lopment trade
Ship Software OnTimetrade
Only $495 for up to 5 Users bull Only $995 for up to 10 UsersFree Single-User Installations
OnTime 2006 Small Team Edition
bull For Teams up to 10 Membersbull Free Single-User Installationsbull $495 for 5-Team Membersbull $995 for 10-Team Members
OnTime 2006 Professional Edition
bull For Teams up to 10 Membersbull Free Single-User Installationsbull $495 for 5-Team Membersbull $995 for 10-Team Members
bull For Teams of 10 to 1000 Membersbull From $149 Per User
SDTimesAd_OnTime2006indd 1 62706 14151 PM
Managing the entire scope of one or more software devel-opment projects can be daunting Within the sea of
tasks features defects and test cycles among team mem-bers managers stakeholders and customers just figuringout whorsquos working on what can be a challenge Thatrsquos where
Axosoftrsquos OnTime 2006enters the picture
OnTime 2006 is adefect management sys-tem designed to help soft-ware development teamsship software on timeWith a focus on ship-ping software on timeit not only effectivelytracks and managesbugs it effectivelyaddresses the broad-er challenges andbest practices ofproject manage-ment
OnTime 2006offers maximum
flexibility for adminis-trators and users alikewith ready access from
a Windows client a Webbrowser or within Visual Studio The highly
configurable and customizable application can also be inte-grated with leading SCM packages including Perforce SCMSourceGear Vault Subversion and Visual SourceSafe
In addition to tracking defects features and tasks thesoftware governs projects with highly customizable workflowand security rules It also offers e-mail notifications and con-versation thread tracking time tracking and work loggingcustom fields and reporting audit trails and archiving
While many of Axosoftrsquos clients switch to OnTime 2006from other products utilizing its importing functionalityothers seek to upgrade from a manual process
ldquoTraditionally development tasks requests defects andother items that occur over the course of the developmentcycle have been tracked in spreadsheets providing ampleopportunity for human error and miscommunicationbetween teamsrdquo says Dan Suceava chief software architectfor Axosoft (wwwaxosoftcom)
ldquoOnTime 2006 keeps track of everything and prevents
important items from slipping through the cracksrdquo heexplains ldquoInstead of wasting developer time with trivialupdates and report requests managers can easily pull theinformation themselves OnTime 2006 frees developers frommost if not all of the process-related overhead that sur-rounds a project and it helps them focus on what they dobest building great softwarerdquo
Developers who use OnTime 2006 see a clear intuitiveview of all of their projects including all the issues defectsfeature requests milestones and tasks As they completetasks developers escalate those items to subsequent work-flow steps This can be set to automatically e-mail otherteam members when the next set of actions is ready to beperformed
OnTime 2006 allows project managers to define speci-fications workflows and security rules They can create ldquowhatifrdquo scenarios that generate predictions for completion timesworkload distribution and other milestones During the pro-jectrsquos execution project managers have access to total proj-ect visibility They know who is working on what the progressbeing made on tasks where bottlenecks are occurring defectrates and estimated completion times
After a project has been completed managers can takeadvantage of the accurate project history that continues toreside within OnTime 2006 This information can be usedfor anything from measuring various productivity rates toproviding a basis for decisions concerning future projectsThus past projects become part of a living knowledge basethat can be consulted at any time
Axosoft even offers a Customer Portal add-on for OnTime2006 that embraces customers as participants in the processof shipping software on time It provides a Web interfacetied to the OnTime database where customers can submit
bugs and other inputCustomizable security set-tings determine how muchproject information will bevisible to customers This
functionality is especially useful for consultants and ISVsduring beta-testing phases
More functionality is in the works for the productldquoOnTime 2006 is already a tool designed to enable the entiredevelopment team to ship software on timerdquo explainsSuceava ldquoWhile today it meets the needs of project man-agers developers and testers extremely well future versionswill provide further functionality for support professionalsIT directors and executivesrdquo ampamp
Axosoft Keeps DevelopmentProjects on the Fast Track
1 october 2006 the artampamp science of software testing u 17
d e f e c t t r a c k i n g
gold sponsor
Dan Suceava
Chief Software Architect
Software configuration management isnrsquot a luxurymdashforany modern software development team itrsquos a necessi-
ty But when an SCM system is as easy to use easy to admin-ister and full of productivity-enhancing features and bene-fits as Perforcersquos system itrsquoll feel like all of your developersare traveling in first class all the way
ldquoThe Perforce SCM System lets teams of local and dis-tributed developers share project files that are centrally storedand managed by the Perforce Serverrdquo says John WalkerPerforcersquos principal product consultant ldquoThe Perforce Serverhandles user requests and tracks all development activity inthe built-in Perforce database Each filersquos state informationcan be quickly discerned from any of Perforcersquos cross-platform clients This means that a user can see when a ver-sion of a particular file has been updated deleted or addedto the server Icons associated with the files indicate whethertheyrsquore currently being edited added or deleted by otherdevelopersrdquo
The Perforce SCM System(wwwperforcecom) letseach developer on your teamobtain and refresh a localprivate copy of versionedfiles by synchronizing themwith files stored in a filedepot Since all of themetadata is centralizedfile state information canbe gathered quickly andeasily Perforce efficient-ly manages both bina-ry and text files fromsource code to graph-ics to documentationThatrsquos one reasonwhy Perforce has
been adopted not onlyby pure developmentshops but also by
chip and hardwaremanufacturers that maintain
large binary assets For example Walker says the Perforce System is very pop-
ular in the game development industry where artists arerequired to create and store large numbers of large imagefiles To help all usersmdashnot just software developersmdashthePerforce software automatically renders thumbnail versions
of image files that can be viewed from within the applica-tionrsquos cross-platform graphical client P4V Regardless ofwhether yoursquore working with text or binary artifacts Perforcehandles the job with style
Perforcersquos clientserverapplication operates overany network or the In-ternet and includes itsown internal database
enginemdashsaving you money improving performance and get-ting the system up and running fast Installing the PerforceServer is simple Download the installation set from the com-panyrsquos Web site and run it Installing any one of a varietyof Perforce clients is also straightforward and the resourcesneeded to manage the installation are minimal Even largedistributed sites of 2000 or more users can be maintainedby a single administrator explains Walker
While Perforce has the basics down pat the real payoff isin the extras that the softwarersquos creators developed to makeyour own coders more efficient Perforcersquos intelligent branch-ing mechanism lets developers work on different releasebranches of a particular application in parallel Code linessupporting specific releases are clearly and visually definedin the system Once a branch is created the relationship itshares with the parent branch is tracked in the serverrsquos meta-data Since the ancestry of files is tracked between branch-es the integration history is maintained in the metadataThat makes it easy to see which changes have and havenrsquotbeen integrated between the related branchesmdashwhether yoursquoreworking in Java J2EE NET CC++ C Visual Basic orHTML
Perforce provides a basic defect-tracking system called jobsA job typically represents an enhancement request or a bugto be fixed Job definitions are customizable to support work-flow and jobs can also work with leading third-party defecttracking systems Support for Mercuryrsquos Quality Center isplanned for the next release This integration will allow usersto enter bugs in Quality Center and have them replicatedinto Perforce as jobs
ldquoThe Perforce SCM Systemrsquos high performance is the prod-uct of a streamlined architecture and closely integrated imple-mentationmdashnot expensive server or network hardwarerdquoWalker says ldquoThe Perforce Server does not require dedicat-ed hardware and client workstations never need upgradingThe systemrsquos networking capabilities arenrsquot a significant loadfor a typical LAN With little need for customization or con-figuration you can be up and running in minutesrdquo ampamp
Perforce Makes SCM FastEasy and First-Class
s o f t w a r e c o n f i g u r a t i o n m a n a g e m e n t
gold sponsor
John Walker
Principal Product Consultant
1 october 2006 the artampamp science of software testing u 19
The faster you find a security problem in the application lifecycle the better If developers catch vulnerabilities early they
can be fixed prior to pushing apps off to QA and customersFixing problems after they slip into a product can cost far morethan catching them early And of course if you donrsquot find the
defects someone else willldquoCenzic solutions provide
tremendous efficiencies andan immediate return on investmentrdquo says JohnWeinschenk Cenzicrsquospresident and CEO (wwwcenziccom) ldquoBy auto-mating the security test-ing process our cus-tomers can secure theirapplications faster andless expensively Likeperformance andfunctionality testingsecurity testingshould be automat-ed Vulnerabilitiesare an open invita-
tion to hackers AllWeb applications andinfrastructure can be
tested with Cenzic solutionsto locate security problemsrdquo
Cenzicrsquos two key security products are Cenzic Hailstorm andCenzic ClickToSecure These solutions provide the most com-prehensive and accurate tests in the industry Weinschenk saysthat while most vendors use scanning technology to focus onlyon commonly known vulnerabilities Hailstorm uses Cenzicrsquospatent-pending Stateful Assessment technology to automatepenetration testing by using a series of transactions to identifyvulnerabilities
Plus Hailstormrsquos SmartAttack Objects Library takes thesoftware beyond merely finding security holesmdashit can also helpenforce internal policies as well as bring organizations intoregulatory compliance with rules like the Gramm-Leach-BlileyAct (GLBA) SB1386 and Sarbanes-Oxley and the PaymentCard Industry (PCI) Data Security Standard and with bestpractices including SANS and OWASP in addition to manyother regulatory standards
ClickToSecure is a service that allows developers to make useof Software as a Service (SaaS) Tests are conducted remotely by
Cenzicrsquos own security expertsmdashand that means that your devel-opment team can build secure software even if it doesnrsquot haveexpertise in that area ClickToSecure accesses applicationsusing a combination of Hailstorm technology and the CenzicIntelligent Analysis (CIA) Research Lab to run assessments
In fact that CIA Lab is the foundation of the SmartAttackObjects Library Cenzic provides continuous updates based onnew vulnerabilities through the lab similar to an anti-virusmodel to help you stay ahead of the exploits and attacks Youcan use the library just as it is plus you can use CenzicrsquosSmartAttack Objects as templates that your own security archi-tects can customize for your organizationrsquos special require-ments if necessary
In addition to its software applications and services Cenzicoffers security training courses that give customers the knowl-edge and skills to use and maintain its products successfullyThe company also offers consulting services that include on-site assessment methodology and software-engineering andconsulting implementation with professional vulnerabilityconsultants who have experience in penetration testing andethical hacking
Among Cenzicrsquos many customers are Boston College DebtExchange IRIS Link and K2 Networks Boston Collegersquos devel-opment group uses Hailstorm in-house throughout its infor-mation security group to test all university Web applicationsAs they find security weaknesses developers use Hailstormreports to remediate those vulnerabilities The same group alsoaddresses its regulatory compliance issues with Hailstorm
In the case of Debt Exchange IRIS Link and K2 NetworksClickToSecure is put to work to perform security assessmentsCustomers call or fax requests to test their applications andCenzic experts test them remotely using Hailstorm in collabo-ration with the CIA Labrsquos expertise Detailed results are then
presented to the customeralong with detailed remedi-ation information
Weinschenk says thatCenzic solutions are a perfect fit for development managersand other executives Using a dashboard customers can viewapplications in the testing phase as well as the number andtypes of vulnerabilities that are found
Weinschenk adds that testing is extremely importantbecause susceptibility to intrusion can result in major recoverycosts and regulatory penalties He also reports that Cenzic hasnt tested a single application that was not vulnerable Butdonrsquot worrymdashCenzic finds the vulnerabilities before the badguys do Put Cenzic to work for you ampamp
Keeping the Bad Guys At Bay With Cenzic Solutions
1 october 2006 the artampamp science of software testing u 21
w e b s e c u r i t y
John Weinschenk
President and Chief Executive Officer
gold sponsor
Donrsquot MissOut
On Another Issue of The
Test amp QAReport
e-newsletter
Each FREE weekly issue includes original articles that interview top thought leaders in software testingand quality trends best practices
and testQA methodologiesGet must-read articles that appear
only in this e-newsletter
Sign up atwwwstpmagcomtqa
Many tools exist that help QA departments and programmerstest applications However itrsquos also crucial that senior-
level software managers have the necessary information to inter-act with executive management and accurately assess the effec-tiveness of their development teams ldquoIf addressing software qual-ity early in the development cycle is an important priority for yourcompany Stelligent can helprdquo says Burke Cox Stelligentrsquos CEO
ldquoAs the thought leaders in early testing we use commercialand open-source technologies to assess and manage softwarequality during the development and assembly of applicationsrdquoCox says ldquoThe Quality Risk Index provides an objective meas-ure of software quality that enables an entire organization toassess quality and progressrdquo
Cox believes that Stelligent (wwwstelligentcom) transformsthe way software is developed and tested By introducing com-prehensive inspection as part of a continuous process both thedevelopment team and senior management gain crucial visibil-ity into the quality risks associated with software projects This
real-time feedback lets organ-izations manage quality longbefore products are everdelivered to the QA team forevaluation
Stelligentrsquos services suchas its Kickstart Quality RiskAssessment are used byorganizations developingsoftware using managedlanguages like C orJava providing bothstatic and dynamicanalysis on the appli-cation source andcompiled code
How effective isthe Kickstart Quality
Risk Assessment Wellone customer recentlyacquired a software
product that it believedwas of high quality based on a
due-diligence assessment of the source code andend user experience However when trying to integrate the prod-uct into its larger solution it found the product had poor toler-ance to change
Thatrsquos when the customer called Stelligent whose KickstartQuality Risk Assessment identified the problems Then its
Quality Roadmap and Seven-Point Implementation Plan identified top priorities for making programmatic changes through refactoring and introduced developer testing as an active part of the continuous integration process The customer
found that Stelligentrsquos QualityRoadmap quickly solved theproblem and then its Con-tinuous Quality program kept
them solved Case closedldquoFor many organizations our Quality Risk Index is the first
objective measure of their software quality that they have everseenrdquo Cox says ldquoThis helps the software manager by providingan independent assessment of what their quality practices cur-rently achieve When working with the executive managementteam in developing schedules approving budgets and other prod-uct management activities the software manager uses Stelligentreports as a basis for discussion For example high code com-plexity with poor test coverage might make a compelling argu-ment to delay release as opposed to simply stating that the prod-uct is just not ready for prime timerdquo
Stelligentrsquos services provide a critical and objective meas-urement of your softwarersquos quality Use this intelligence tomake it difficult to enter new defects into the source repos-itory helping your developers leverage the continuous qual-ity feedback before code even reaches the QA department
Stelligent is a subsidiary of JNetDirect which makesConvergence a solution for providing real-time visibility intosoftware quality CoView which helps your team develop accu-rate and effective JUnit tests and also high-performance JDBCdrivers But long before JNetDirect acquired Stelligent it was acustomer using the Kickstart Quality Risk Assessment servicesto help build its own products Stelligentrsquos impact was so dra-matic that JNetDirect immediately saw the value of bringingStelligentrsquos services to a broader audience
ldquoDefects will always enter the build systemrdquo Cox says ldquoOurbusiness is ensuring that they cannot live there for very longQuality may not have a material impact on initial product rev-enue but it is the most significant driver of product profitabil-ity Most companies struggle with describing quality it is mixedwith subjective measures and anecdotal evidence Our customershave a quality score they can point to when setting goals andbenchmarks For organizations interested in increasing earningslowering the costs associated with poor product quality is thebest place to startrdquo
Stelligentrsquos services start with the Kickstart Quality RiskAssessmentmdashask the company how its objective measures canhelp you too improve your software quality ampamp
Stelligent Brings ObjectivityTo Quality Measurement
t e s t i n g s e r v i c e s
Burke Cox
Chief Executive Officer
1 october 2006 the artampamp science of software testing u 23
silver sponsor
The Fast amp Scalable Team Solution forDefect amp Issue Tracking bull Feature amp Change Tracking bull Task amp To-do List Tracking bull Helpdesk Ticket Tracking
OnTime is the market-leading project defect and feature management tool for agile software development and test teamsOnTime facilitates tracking analyzing and trending team-based software development efforts in an intuitive and powerful user interface A fully customizable UI powerful workflow process enforcements two-way email communications and custom reports combine to help software development teams ship software on-time
Available for Windows Web amp VSNET 20032005
800middot653middot0024
wwwaxosoftcomso f tware fo r so f tware deve lopment trade
Ship Software OnTimetrade
Only $495 for up to 5 Users bull Only $995 for up to 10 UsersFree Single-User Installations
OnTime 2006 Small Team Edition
bull For Teams up to 10 Membersbull Free Single-User Installationsbull $495 for 5-Team Membersbull $995 for 10-Team Members
OnTime 2006 Professional Edition
bull For Teams up to 10 Membersbull Free Single-User Installationsbull $495 for 5-Team Membersbull $995 for 10-Team Members
bull For Teams of 10 to 1000 Membersbull From $149 Per User
SDTimesAd_OnTime2006indd 1 62706 14151 PM
Managing the entire scope of one or more software devel-opment projects can be daunting Within the sea of
tasks features defects and test cycles among team mem-bers managers stakeholders and customers just figuringout whorsquos working on what can be a challenge Thatrsquos where
Axosoftrsquos OnTime 2006enters the picture
OnTime 2006 is adefect management sys-tem designed to help soft-ware development teamsship software on timeWith a focus on ship-ping software on timeit not only effectivelytracks and managesbugs it effectivelyaddresses the broad-er challenges andbest practices ofproject manage-ment
OnTime 2006offers maximum
flexibility for adminis-trators and users alikewith ready access from
a Windows client a Webbrowser or within Visual Studio The highly
configurable and customizable application can also be inte-grated with leading SCM packages including Perforce SCMSourceGear Vault Subversion and Visual SourceSafe
In addition to tracking defects features and tasks thesoftware governs projects with highly customizable workflowand security rules It also offers e-mail notifications and con-versation thread tracking time tracking and work loggingcustom fields and reporting audit trails and archiving
While many of Axosoftrsquos clients switch to OnTime 2006from other products utilizing its importing functionalityothers seek to upgrade from a manual process
ldquoTraditionally development tasks requests defects andother items that occur over the course of the developmentcycle have been tracked in spreadsheets providing ampleopportunity for human error and miscommunicationbetween teamsrdquo says Dan Suceava chief software architectfor Axosoft (wwwaxosoftcom)
ldquoOnTime 2006 keeps track of everything and prevents
important items from slipping through the cracksrdquo heexplains ldquoInstead of wasting developer time with trivialupdates and report requests managers can easily pull theinformation themselves OnTime 2006 frees developers frommost if not all of the process-related overhead that sur-rounds a project and it helps them focus on what they dobest building great softwarerdquo
Developers who use OnTime 2006 see a clear intuitiveview of all of their projects including all the issues defectsfeature requests milestones and tasks As they completetasks developers escalate those items to subsequent work-flow steps This can be set to automatically e-mail otherteam members when the next set of actions is ready to beperformed
OnTime 2006 allows project managers to define speci-fications workflows and security rules They can create ldquowhatifrdquo scenarios that generate predictions for completion timesworkload distribution and other milestones During the pro-jectrsquos execution project managers have access to total proj-ect visibility They know who is working on what the progressbeing made on tasks where bottlenecks are occurring defectrates and estimated completion times
After a project has been completed managers can takeadvantage of the accurate project history that continues toreside within OnTime 2006 This information can be usedfor anything from measuring various productivity rates toproviding a basis for decisions concerning future projectsThus past projects become part of a living knowledge basethat can be consulted at any time
Axosoft even offers a Customer Portal add-on for OnTime2006 that embraces customers as participants in the processof shipping software on time It provides a Web interfacetied to the OnTime database where customers can submit
bugs and other inputCustomizable security set-tings determine how muchproject information will bevisible to customers This
functionality is especially useful for consultants and ISVsduring beta-testing phases
More functionality is in the works for the productldquoOnTime 2006 is already a tool designed to enable the entiredevelopment team to ship software on timerdquo explainsSuceava ldquoWhile today it meets the needs of project man-agers developers and testers extremely well future versionswill provide further functionality for support professionalsIT directors and executivesrdquo ampamp
Axosoft Keeps DevelopmentProjects on the Fast Track
1 october 2006 the artampamp science of software testing u 17
d e f e c t t r a c k i n g
gold sponsor
Dan Suceava
Chief Software Architect
Software configuration management isnrsquot a luxurymdashforany modern software development team itrsquos a necessi-
ty But when an SCM system is as easy to use easy to admin-ister and full of productivity-enhancing features and bene-fits as Perforcersquos system itrsquoll feel like all of your developersare traveling in first class all the way
ldquoThe Perforce SCM System lets teams of local and dis-tributed developers share project files that are centrally storedand managed by the Perforce Serverrdquo says John WalkerPerforcersquos principal product consultant ldquoThe Perforce Serverhandles user requests and tracks all development activity inthe built-in Perforce database Each filersquos state informationcan be quickly discerned from any of Perforcersquos cross-platform clients This means that a user can see when a ver-sion of a particular file has been updated deleted or addedto the server Icons associated with the files indicate whethertheyrsquore currently being edited added or deleted by otherdevelopersrdquo
The Perforce SCM System(wwwperforcecom) letseach developer on your teamobtain and refresh a localprivate copy of versionedfiles by synchronizing themwith files stored in a filedepot Since all of themetadata is centralizedfile state information canbe gathered quickly andeasily Perforce efficient-ly manages both bina-ry and text files fromsource code to graph-ics to documentationThatrsquos one reasonwhy Perforce has
been adopted not onlyby pure developmentshops but also by
chip and hardwaremanufacturers that maintain
large binary assets For example Walker says the Perforce System is very pop-
ular in the game development industry where artists arerequired to create and store large numbers of large imagefiles To help all usersmdashnot just software developersmdashthePerforce software automatically renders thumbnail versions
of image files that can be viewed from within the applica-tionrsquos cross-platform graphical client P4V Regardless ofwhether yoursquore working with text or binary artifacts Perforcehandles the job with style
Perforcersquos clientserverapplication operates overany network or the In-ternet and includes itsown internal database
enginemdashsaving you money improving performance and get-ting the system up and running fast Installing the PerforceServer is simple Download the installation set from the com-panyrsquos Web site and run it Installing any one of a varietyof Perforce clients is also straightforward and the resourcesneeded to manage the installation are minimal Even largedistributed sites of 2000 or more users can be maintainedby a single administrator explains Walker
While Perforce has the basics down pat the real payoff isin the extras that the softwarersquos creators developed to makeyour own coders more efficient Perforcersquos intelligent branch-ing mechanism lets developers work on different releasebranches of a particular application in parallel Code linessupporting specific releases are clearly and visually definedin the system Once a branch is created the relationship itshares with the parent branch is tracked in the serverrsquos meta-data Since the ancestry of files is tracked between branch-es the integration history is maintained in the metadataThat makes it easy to see which changes have and havenrsquotbeen integrated between the related branchesmdashwhether yoursquoreworking in Java J2EE NET CC++ C Visual Basic orHTML
Perforce provides a basic defect-tracking system called jobsA job typically represents an enhancement request or a bugto be fixed Job definitions are customizable to support work-flow and jobs can also work with leading third-party defecttracking systems Support for Mercuryrsquos Quality Center isplanned for the next release This integration will allow usersto enter bugs in Quality Center and have them replicatedinto Perforce as jobs
ldquoThe Perforce SCM Systemrsquos high performance is the prod-uct of a streamlined architecture and closely integrated imple-mentationmdashnot expensive server or network hardwarerdquoWalker says ldquoThe Perforce Server does not require dedicat-ed hardware and client workstations never need upgradingThe systemrsquos networking capabilities arenrsquot a significant loadfor a typical LAN With little need for customization or con-figuration you can be up and running in minutesrdquo ampamp
Perforce Makes SCM FastEasy and First-Class
s o f t w a r e c o n f i g u r a t i o n m a n a g e m e n t
gold sponsor
John Walker
Principal Product Consultant
1 october 2006 the artampamp science of software testing u 19
The faster you find a security problem in the application lifecycle the better If developers catch vulnerabilities early they
can be fixed prior to pushing apps off to QA and customersFixing problems after they slip into a product can cost far morethan catching them early And of course if you donrsquot find the
defects someone else willldquoCenzic solutions provide
tremendous efficiencies andan immediate return on investmentrdquo says JohnWeinschenk Cenzicrsquospresident and CEO (wwwcenziccom) ldquoBy auto-mating the security test-ing process our cus-tomers can secure theirapplications faster andless expensively Likeperformance andfunctionality testingsecurity testingshould be automat-ed Vulnerabilitiesare an open invita-
tion to hackers AllWeb applications andinfrastructure can be
tested with Cenzic solutionsto locate security problemsrdquo
Cenzicrsquos two key security products are Cenzic Hailstorm andCenzic ClickToSecure These solutions provide the most com-prehensive and accurate tests in the industry Weinschenk saysthat while most vendors use scanning technology to focus onlyon commonly known vulnerabilities Hailstorm uses Cenzicrsquospatent-pending Stateful Assessment technology to automatepenetration testing by using a series of transactions to identifyvulnerabilities
Plus Hailstormrsquos SmartAttack Objects Library takes thesoftware beyond merely finding security holesmdashit can also helpenforce internal policies as well as bring organizations intoregulatory compliance with rules like the Gramm-Leach-BlileyAct (GLBA) SB1386 and Sarbanes-Oxley and the PaymentCard Industry (PCI) Data Security Standard and with bestpractices including SANS and OWASP in addition to manyother regulatory standards
ClickToSecure is a service that allows developers to make useof Software as a Service (SaaS) Tests are conducted remotely by
Cenzicrsquos own security expertsmdashand that means that your devel-opment team can build secure software even if it doesnrsquot haveexpertise in that area ClickToSecure accesses applicationsusing a combination of Hailstorm technology and the CenzicIntelligent Analysis (CIA) Research Lab to run assessments
In fact that CIA Lab is the foundation of the SmartAttackObjects Library Cenzic provides continuous updates based onnew vulnerabilities through the lab similar to an anti-virusmodel to help you stay ahead of the exploits and attacks Youcan use the library just as it is plus you can use CenzicrsquosSmartAttack Objects as templates that your own security archi-tects can customize for your organizationrsquos special require-ments if necessary
In addition to its software applications and services Cenzicoffers security training courses that give customers the knowl-edge and skills to use and maintain its products successfullyThe company also offers consulting services that include on-site assessment methodology and software-engineering andconsulting implementation with professional vulnerabilityconsultants who have experience in penetration testing andethical hacking
Among Cenzicrsquos many customers are Boston College DebtExchange IRIS Link and K2 Networks Boston Collegersquos devel-opment group uses Hailstorm in-house throughout its infor-mation security group to test all university Web applicationsAs they find security weaknesses developers use Hailstormreports to remediate those vulnerabilities The same group alsoaddresses its regulatory compliance issues with Hailstorm
In the case of Debt Exchange IRIS Link and K2 NetworksClickToSecure is put to work to perform security assessmentsCustomers call or fax requests to test their applications andCenzic experts test them remotely using Hailstorm in collabo-ration with the CIA Labrsquos expertise Detailed results are then
presented to the customeralong with detailed remedi-ation information
Weinschenk says thatCenzic solutions are a perfect fit for development managersand other executives Using a dashboard customers can viewapplications in the testing phase as well as the number andtypes of vulnerabilities that are found
Weinschenk adds that testing is extremely importantbecause susceptibility to intrusion can result in major recoverycosts and regulatory penalties He also reports that Cenzic hasnt tested a single application that was not vulnerable Butdonrsquot worrymdashCenzic finds the vulnerabilities before the badguys do Put Cenzic to work for you ampamp
Keeping the Bad Guys At Bay With Cenzic Solutions
1 october 2006 the artampamp science of software testing u 21
w e b s e c u r i t y
John Weinschenk
President and Chief Executive Officer
gold sponsor
Donrsquot MissOut
On Another Issue of The
Test amp QAReport
e-newsletter
Each FREE weekly issue includes original articles that interview top thought leaders in software testingand quality trends best practices
and testQA methodologiesGet must-read articles that appear
only in this e-newsletter
Sign up atwwwstpmagcomtqa
Many tools exist that help QA departments and programmerstest applications However itrsquos also crucial that senior-
level software managers have the necessary information to inter-act with executive management and accurately assess the effec-tiveness of their development teams ldquoIf addressing software qual-ity early in the development cycle is an important priority for yourcompany Stelligent can helprdquo says Burke Cox Stelligentrsquos CEO
ldquoAs the thought leaders in early testing we use commercialand open-source technologies to assess and manage softwarequality during the development and assembly of applicationsrdquoCox says ldquoThe Quality Risk Index provides an objective meas-ure of software quality that enables an entire organization toassess quality and progressrdquo
Cox believes that Stelligent (wwwstelligentcom) transformsthe way software is developed and tested By introducing com-prehensive inspection as part of a continuous process both thedevelopment team and senior management gain crucial visibil-ity into the quality risks associated with software projects This
real-time feedback lets organ-izations manage quality longbefore products are everdelivered to the QA team forevaluation
Stelligentrsquos services suchas its Kickstart Quality RiskAssessment are used byorganizations developingsoftware using managedlanguages like C orJava providing bothstatic and dynamicanalysis on the appli-cation source andcompiled code
How effective isthe Kickstart Quality
Risk Assessment Wellone customer recentlyacquired a software
product that it believedwas of high quality based on a
due-diligence assessment of the source code andend user experience However when trying to integrate the prod-uct into its larger solution it found the product had poor toler-ance to change
Thatrsquos when the customer called Stelligent whose KickstartQuality Risk Assessment identified the problems Then its
Quality Roadmap and Seven-Point Implementation Plan identified top priorities for making programmatic changes through refactoring and introduced developer testing as an active part of the continuous integration process The customer
found that Stelligentrsquos QualityRoadmap quickly solved theproblem and then its Con-tinuous Quality program kept
them solved Case closedldquoFor many organizations our Quality Risk Index is the first
objective measure of their software quality that they have everseenrdquo Cox says ldquoThis helps the software manager by providingan independent assessment of what their quality practices cur-rently achieve When working with the executive managementteam in developing schedules approving budgets and other prod-uct management activities the software manager uses Stelligentreports as a basis for discussion For example high code com-plexity with poor test coverage might make a compelling argu-ment to delay release as opposed to simply stating that the prod-uct is just not ready for prime timerdquo
Stelligentrsquos services provide a critical and objective meas-urement of your softwarersquos quality Use this intelligence tomake it difficult to enter new defects into the source repos-itory helping your developers leverage the continuous qual-ity feedback before code even reaches the QA department
Stelligent is a subsidiary of JNetDirect which makesConvergence a solution for providing real-time visibility intosoftware quality CoView which helps your team develop accu-rate and effective JUnit tests and also high-performance JDBCdrivers But long before JNetDirect acquired Stelligent it was acustomer using the Kickstart Quality Risk Assessment servicesto help build its own products Stelligentrsquos impact was so dra-matic that JNetDirect immediately saw the value of bringingStelligentrsquos services to a broader audience
ldquoDefects will always enter the build systemrdquo Cox says ldquoOurbusiness is ensuring that they cannot live there for very longQuality may not have a material impact on initial product rev-enue but it is the most significant driver of product profitabil-ity Most companies struggle with describing quality it is mixedwith subjective measures and anecdotal evidence Our customershave a quality score they can point to when setting goals andbenchmarks For organizations interested in increasing earningslowering the costs associated with poor product quality is thebest place to startrdquo
Stelligentrsquos services start with the Kickstart Quality RiskAssessmentmdashask the company how its objective measures canhelp you too improve your software quality ampamp
Stelligent Brings ObjectivityTo Quality Measurement
t e s t i n g s e r v i c e s
Burke Cox
Chief Executive Officer
1 october 2006 the artampamp science of software testing u 23
silver sponsor
Managing the entire scope of one or more software devel-opment projects can be daunting Within the sea of
tasks features defects and test cycles among team mem-bers managers stakeholders and customers just figuringout whorsquos working on what can be a challenge Thatrsquos where
Axosoftrsquos OnTime 2006enters the picture
OnTime 2006 is adefect management sys-tem designed to help soft-ware development teamsship software on timeWith a focus on ship-ping software on timeit not only effectivelytracks and managesbugs it effectivelyaddresses the broad-er challenges andbest practices ofproject manage-ment
OnTime 2006offers maximum
flexibility for adminis-trators and users alikewith ready access from
a Windows client a Webbrowser or within Visual Studio The highly
configurable and customizable application can also be inte-grated with leading SCM packages including Perforce SCMSourceGear Vault Subversion and Visual SourceSafe
In addition to tracking defects features and tasks thesoftware governs projects with highly customizable workflowand security rules It also offers e-mail notifications and con-versation thread tracking time tracking and work loggingcustom fields and reporting audit trails and archiving
While many of Axosoftrsquos clients switch to OnTime 2006from other products utilizing its importing functionalityothers seek to upgrade from a manual process
ldquoTraditionally development tasks requests defects andother items that occur over the course of the developmentcycle have been tracked in spreadsheets providing ampleopportunity for human error and miscommunicationbetween teamsrdquo says Dan Suceava chief software architectfor Axosoft (wwwaxosoftcom)
ldquoOnTime 2006 keeps track of everything and prevents
important items from slipping through the cracksrdquo heexplains ldquoInstead of wasting developer time with trivialupdates and report requests managers can easily pull theinformation themselves OnTime 2006 frees developers frommost if not all of the process-related overhead that sur-rounds a project and it helps them focus on what they dobest building great softwarerdquo
Developers who use OnTime 2006 see a clear intuitiveview of all of their projects including all the issues defectsfeature requests milestones and tasks As they completetasks developers escalate those items to subsequent work-flow steps This can be set to automatically e-mail otherteam members when the next set of actions is ready to beperformed
OnTime 2006 allows project managers to define speci-fications workflows and security rules They can create ldquowhatifrdquo scenarios that generate predictions for completion timesworkload distribution and other milestones During the pro-jectrsquos execution project managers have access to total proj-ect visibility They know who is working on what the progressbeing made on tasks where bottlenecks are occurring defectrates and estimated completion times
After a project has been completed managers can takeadvantage of the accurate project history that continues toreside within OnTime 2006 This information can be usedfor anything from measuring various productivity rates toproviding a basis for decisions concerning future projectsThus past projects become part of a living knowledge basethat can be consulted at any time
Axosoft even offers a Customer Portal add-on for OnTime2006 that embraces customers as participants in the processof shipping software on time It provides a Web interfacetied to the OnTime database where customers can submit
bugs and other inputCustomizable security set-tings determine how muchproject information will bevisible to customers This
functionality is especially useful for consultants and ISVsduring beta-testing phases
More functionality is in the works for the productldquoOnTime 2006 is already a tool designed to enable the entiredevelopment team to ship software on timerdquo explainsSuceava ldquoWhile today it meets the needs of project man-agers developers and testers extremely well future versionswill provide further functionality for support professionalsIT directors and executivesrdquo ampamp
Axosoft Keeps DevelopmentProjects on the Fast Track
1 october 2006 the artampamp science of software testing u 17
d e f e c t t r a c k i n g
gold sponsor
Dan Suceava
Chief Software Architect
Software configuration management isnrsquot a luxurymdashforany modern software development team itrsquos a necessi-
ty But when an SCM system is as easy to use easy to admin-ister and full of productivity-enhancing features and bene-fits as Perforcersquos system itrsquoll feel like all of your developersare traveling in first class all the way
ldquoThe Perforce SCM System lets teams of local and dis-tributed developers share project files that are centrally storedand managed by the Perforce Serverrdquo says John WalkerPerforcersquos principal product consultant ldquoThe Perforce Serverhandles user requests and tracks all development activity inthe built-in Perforce database Each filersquos state informationcan be quickly discerned from any of Perforcersquos cross-platform clients This means that a user can see when a ver-sion of a particular file has been updated deleted or addedto the server Icons associated with the files indicate whethertheyrsquore currently being edited added or deleted by otherdevelopersrdquo
The Perforce SCM System(wwwperforcecom) letseach developer on your teamobtain and refresh a localprivate copy of versionedfiles by synchronizing themwith files stored in a filedepot Since all of themetadata is centralizedfile state information canbe gathered quickly andeasily Perforce efficient-ly manages both bina-ry and text files fromsource code to graph-ics to documentationThatrsquos one reasonwhy Perforce has
been adopted not onlyby pure developmentshops but also by
chip and hardwaremanufacturers that maintain
large binary assets For example Walker says the Perforce System is very pop-
ular in the game development industry where artists arerequired to create and store large numbers of large imagefiles To help all usersmdashnot just software developersmdashthePerforce software automatically renders thumbnail versions
of image files that can be viewed from within the applica-tionrsquos cross-platform graphical client P4V Regardless ofwhether yoursquore working with text or binary artifacts Perforcehandles the job with style
Perforcersquos clientserverapplication operates overany network or the In-ternet and includes itsown internal database
enginemdashsaving you money improving performance and get-ting the system up and running fast Installing the PerforceServer is simple Download the installation set from the com-panyrsquos Web site and run it Installing any one of a varietyof Perforce clients is also straightforward and the resourcesneeded to manage the installation are minimal Even largedistributed sites of 2000 or more users can be maintainedby a single administrator explains Walker
While Perforce has the basics down pat the real payoff isin the extras that the softwarersquos creators developed to makeyour own coders more efficient Perforcersquos intelligent branch-ing mechanism lets developers work on different releasebranches of a particular application in parallel Code linessupporting specific releases are clearly and visually definedin the system Once a branch is created the relationship itshares with the parent branch is tracked in the serverrsquos meta-data Since the ancestry of files is tracked between branch-es the integration history is maintained in the metadataThat makes it easy to see which changes have and havenrsquotbeen integrated between the related branchesmdashwhether yoursquoreworking in Java J2EE NET CC++ C Visual Basic orHTML
Perforce provides a basic defect-tracking system called jobsA job typically represents an enhancement request or a bugto be fixed Job definitions are customizable to support work-flow and jobs can also work with leading third-party defecttracking systems Support for Mercuryrsquos Quality Center isplanned for the next release This integration will allow usersto enter bugs in Quality Center and have them replicatedinto Perforce as jobs
ldquoThe Perforce SCM Systemrsquos high performance is the prod-uct of a streamlined architecture and closely integrated imple-mentationmdashnot expensive server or network hardwarerdquoWalker says ldquoThe Perforce Server does not require dedicat-ed hardware and client workstations never need upgradingThe systemrsquos networking capabilities arenrsquot a significant loadfor a typical LAN With little need for customization or con-figuration you can be up and running in minutesrdquo ampamp
Perforce Makes SCM FastEasy and First-Class
s o f t w a r e c o n f i g u r a t i o n m a n a g e m e n t
gold sponsor
John Walker
Principal Product Consultant
1 october 2006 the artampamp science of software testing u 19
The faster you find a security problem in the application lifecycle the better If developers catch vulnerabilities early they
can be fixed prior to pushing apps off to QA and customersFixing problems after they slip into a product can cost far morethan catching them early And of course if you donrsquot find the
defects someone else willldquoCenzic solutions provide
tremendous efficiencies andan immediate return on investmentrdquo says JohnWeinschenk Cenzicrsquospresident and CEO (wwwcenziccom) ldquoBy auto-mating the security test-ing process our cus-tomers can secure theirapplications faster andless expensively Likeperformance andfunctionality testingsecurity testingshould be automat-ed Vulnerabilitiesare an open invita-
tion to hackers AllWeb applications andinfrastructure can be
tested with Cenzic solutionsto locate security problemsrdquo
Cenzicrsquos two key security products are Cenzic Hailstorm andCenzic ClickToSecure These solutions provide the most com-prehensive and accurate tests in the industry Weinschenk saysthat while most vendors use scanning technology to focus onlyon commonly known vulnerabilities Hailstorm uses Cenzicrsquospatent-pending Stateful Assessment technology to automatepenetration testing by using a series of transactions to identifyvulnerabilities
Plus Hailstormrsquos SmartAttack Objects Library takes thesoftware beyond merely finding security holesmdashit can also helpenforce internal policies as well as bring organizations intoregulatory compliance with rules like the Gramm-Leach-BlileyAct (GLBA) SB1386 and Sarbanes-Oxley and the PaymentCard Industry (PCI) Data Security Standard and with bestpractices including SANS and OWASP in addition to manyother regulatory standards
ClickToSecure is a service that allows developers to make useof Software as a Service (SaaS) Tests are conducted remotely by
Cenzicrsquos own security expertsmdashand that means that your devel-opment team can build secure software even if it doesnrsquot haveexpertise in that area ClickToSecure accesses applicationsusing a combination of Hailstorm technology and the CenzicIntelligent Analysis (CIA) Research Lab to run assessments
In fact that CIA Lab is the foundation of the SmartAttackObjects Library Cenzic provides continuous updates based onnew vulnerabilities through the lab similar to an anti-virusmodel to help you stay ahead of the exploits and attacks Youcan use the library just as it is plus you can use CenzicrsquosSmartAttack Objects as templates that your own security archi-tects can customize for your organizationrsquos special require-ments if necessary
In addition to its software applications and services Cenzicoffers security training courses that give customers the knowl-edge and skills to use and maintain its products successfullyThe company also offers consulting services that include on-site assessment methodology and software-engineering andconsulting implementation with professional vulnerabilityconsultants who have experience in penetration testing andethical hacking
Among Cenzicrsquos many customers are Boston College DebtExchange IRIS Link and K2 Networks Boston Collegersquos devel-opment group uses Hailstorm in-house throughout its infor-mation security group to test all university Web applicationsAs they find security weaknesses developers use Hailstormreports to remediate those vulnerabilities The same group alsoaddresses its regulatory compliance issues with Hailstorm
In the case of Debt Exchange IRIS Link and K2 NetworksClickToSecure is put to work to perform security assessmentsCustomers call or fax requests to test their applications andCenzic experts test them remotely using Hailstorm in collabo-ration with the CIA Labrsquos expertise Detailed results are then
presented to the customeralong with detailed remedi-ation information
Weinschenk says thatCenzic solutions are a perfect fit for development managersand other executives Using a dashboard customers can viewapplications in the testing phase as well as the number andtypes of vulnerabilities that are found
Weinschenk adds that testing is extremely importantbecause susceptibility to intrusion can result in major recoverycosts and regulatory penalties He also reports that Cenzic hasnt tested a single application that was not vulnerable Butdonrsquot worrymdashCenzic finds the vulnerabilities before the badguys do Put Cenzic to work for you ampamp
Keeping the Bad Guys At Bay With Cenzic Solutions
1 october 2006 the artampamp science of software testing u 21
w e b s e c u r i t y
John Weinschenk
President and Chief Executive Officer
gold sponsor
Donrsquot MissOut
On Another Issue of The
Test amp QAReport
e-newsletter
Each FREE weekly issue includes original articles that interview top thought leaders in software testingand quality trends best practices
and testQA methodologiesGet must-read articles that appear
only in this e-newsletter
Sign up atwwwstpmagcomtqa
Many tools exist that help QA departments and programmerstest applications However itrsquos also crucial that senior-
level software managers have the necessary information to inter-act with executive management and accurately assess the effec-tiveness of their development teams ldquoIf addressing software qual-ity early in the development cycle is an important priority for yourcompany Stelligent can helprdquo says Burke Cox Stelligentrsquos CEO
ldquoAs the thought leaders in early testing we use commercialand open-source technologies to assess and manage softwarequality during the development and assembly of applicationsrdquoCox says ldquoThe Quality Risk Index provides an objective meas-ure of software quality that enables an entire organization toassess quality and progressrdquo
Cox believes that Stelligent (wwwstelligentcom) transformsthe way software is developed and tested By introducing com-prehensive inspection as part of a continuous process both thedevelopment team and senior management gain crucial visibil-ity into the quality risks associated with software projects This
real-time feedback lets organ-izations manage quality longbefore products are everdelivered to the QA team forevaluation
Stelligentrsquos services suchas its Kickstart Quality RiskAssessment are used byorganizations developingsoftware using managedlanguages like C orJava providing bothstatic and dynamicanalysis on the appli-cation source andcompiled code
How effective isthe Kickstart Quality
Risk Assessment Wellone customer recentlyacquired a software
product that it believedwas of high quality based on a
due-diligence assessment of the source code andend user experience However when trying to integrate the prod-uct into its larger solution it found the product had poor toler-ance to change
Thatrsquos when the customer called Stelligent whose KickstartQuality Risk Assessment identified the problems Then its
Quality Roadmap and Seven-Point Implementation Plan identified top priorities for making programmatic changes through refactoring and introduced developer testing as an active part of the continuous integration process The customer
found that Stelligentrsquos QualityRoadmap quickly solved theproblem and then its Con-tinuous Quality program kept
them solved Case closedldquoFor many organizations our Quality Risk Index is the first
objective measure of their software quality that they have everseenrdquo Cox says ldquoThis helps the software manager by providingan independent assessment of what their quality practices cur-rently achieve When working with the executive managementteam in developing schedules approving budgets and other prod-uct management activities the software manager uses Stelligentreports as a basis for discussion For example high code com-plexity with poor test coverage might make a compelling argu-ment to delay release as opposed to simply stating that the prod-uct is just not ready for prime timerdquo
Stelligentrsquos services provide a critical and objective meas-urement of your softwarersquos quality Use this intelligence tomake it difficult to enter new defects into the source repos-itory helping your developers leverage the continuous qual-ity feedback before code even reaches the QA department
Stelligent is a subsidiary of JNetDirect which makesConvergence a solution for providing real-time visibility intosoftware quality CoView which helps your team develop accu-rate and effective JUnit tests and also high-performance JDBCdrivers But long before JNetDirect acquired Stelligent it was acustomer using the Kickstart Quality Risk Assessment servicesto help build its own products Stelligentrsquos impact was so dra-matic that JNetDirect immediately saw the value of bringingStelligentrsquos services to a broader audience
ldquoDefects will always enter the build systemrdquo Cox says ldquoOurbusiness is ensuring that they cannot live there for very longQuality may not have a material impact on initial product rev-enue but it is the most significant driver of product profitabil-ity Most companies struggle with describing quality it is mixedwith subjective measures and anecdotal evidence Our customershave a quality score they can point to when setting goals andbenchmarks For organizations interested in increasing earningslowering the costs associated with poor product quality is thebest place to startrdquo
Stelligentrsquos services start with the Kickstart Quality RiskAssessmentmdashask the company how its objective measures canhelp you too improve your software quality ampamp
Stelligent Brings ObjectivityTo Quality Measurement
t e s t i n g s e r v i c e s
Burke Cox
Chief Executive Officer
1 october 2006 the artampamp science of software testing u 23
silver sponsor
Software configuration management isnrsquot a luxurymdashforany modern software development team itrsquos a necessi-
ty But when an SCM system is as easy to use easy to admin-ister and full of productivity-enhancing features and bene-fits as Perforcersquos system itrsquoll feel like all of your developersare traveling in first class all the way
ldquoThe Perforce SCM System lets teams of local and dis-tributed developers share project files that are centrally storedand managed by the Perforce Serverrdquo says John WalkerPerforcersquos principal product consultant ldquoThe Perforce Serverhandles user requests and tracks all development activity inthe built-in Perforce database Each filersquos state informationcan be quickly discerned from any of Perforcersquos cross-platform clients This means that a user can see when a ver-sion of a particular file has been updated deleted or addedto the server Icons associated with the files indicate whethertheyrsquore currently being edited added or deleted by otherdevelopersrdquo
The Perforce SCM System(wwwperforcecom) letseach developer on your teamobtain and refresh a localprivate copy of versionedfiles by synchronizing themwith files stored in a filedepot Since all of themetadata is centralizedfile state information canbe gathered quickly andeasily Perforce efficient-ly manages both bina-ry and text files fromsource code to graph-ics to documentationThatrsquos one reasonwhy Perforce has
been adopted not onlyby pure developmentshops but also by
chip and hardwaremanufacturers that maintain
large binary assets For example Walker says the Perforce System is very pop-
ular in the game development industry where artists arerequired to create and store large numbers of large imagefiles To help all usersmdashnot just software developersmdashthePerforce software automatically renders thumbnail versions
of image files that can be viewed from within the applica-tionrsquos cross-platform graphical client P4V Regardless ofwhether yoursquore working with text or binary artifacts Perforcehandles the job with style
Perforcersquos clientserverapplication operates overany network or the In-ternet and includes itsown internal database
enginemdashsaving you money improving performance and get-ting the system up and running fast Installing the PerforceServer is simple Download the installation set from the com-panyrsquos Web site and run it Installing any one of a varietyof Perforce clients is also straightforward and the resourcesneeded to manage the installation are minimal Even largedistributed sites of 2000 or more users can be maintainedby a single administrator explains Walker
While Perforce has the basics down pat the real payoff isin the extras that the softwarersquos creators developed to makeyour own coders more efficient Perforcersquos intelligent branch-ing mechanism lets developers work on different releasebranches of a particular application in parallel Code linessupporting specific releases are clearly and visually definedin the system Once a branch is created the relationship itshares with the parent branch is tracked in the serverrsquos meta-data Since the ancestry of files is tracked between branch-es the integration history is maintained in the metadataThat makes it easy to see which changes have and havenrsquotbeen integrated between the related branchesmdashwhether yoursquoreworking in Java J2EE NET CC++ C Visual Basic orHTML
Perforce provides a basic defect-tracking system called jobsA job typically represents an enhancement request or a bugto be fixed Job definitions are customizable to support work-flow and jobs can also work with leading third-party defecttracking systems Support for Mercuryrsquos Quality Center isplanned for the next release This integration will allow usersto enter bugs in Quality Center and have them replicatedinto Perforce as jobs
ldquoThe Perforce SCM Systemrsquos high performance is the prod-uct of a streamlined architecture and closely integrated imple-mentationmdashnot expensive server or network hardwarerdquoWalker says ldquoThe Perforce Server does not require dedicat-ed hardware and client workstations never need upgradingThe systemrsquos networking capabilities arenrsquot a significant loadfor a typical LAN With little need for customization or con-figuration you can be up and running in minutesrdquo ampamp
Perforce Makes SCM FastEasy and First-Class
s o f t w a r e c o n f i g u r a t i o n m a n a g e m e n t
gold sponsor
John Walker
Principal Product Consultant
1 october 2006 the artampamp science of software testing u 19
The faster you find a security problem in the application lifecycle the better If developers catch vulnerabilities early they
can be fixed prior to pushing apps off to QA and customersFixing problems after they slip into a product can cost far morethan catching them early And of course if you donrsquot find the
defects someone else willldquoCenzic solutions provide
tremendous efficiencies andan immediate return on investmentrdquo says JohnWeinschenk Cenzicrsquospresident and CEO (wwwcenziccom) ldquoBy auto-mating the security test-ing process our cus-tomers can secure theirapplications faster andless expensively Likeperformance andfunctionality testingsecurity testingshould be automat-ed Vulnerabilitiesare an open invita-
tion to hackers AllWeb applications andinfrastructure can be
tested with Cenzic solutionsto locate security problemsrdquo
Cenzicrsquos two key security products are Cenzic Hailstorm andCenzic ClickToSecure These solutions provide the most com-prehensive and accurate tests in the industry Weinschenk saysthat while most vendors use scanning technology to focus onlyon commonly known vulnerabilities Hailstorm uses Cenzicrsquospatent-pending Stateful Assessment technology to automatepenetration testing by using a series of transactions to identifyvulnerabilities
Plus Hailstormrsquos SmartAttack Objects Library takes thesoftware beyond merely finding security holesmdashit can also helpenforce internal policies as well as bring organizations intoregulatory compliance with rules like the Gramm-Leach-BlileyAct (GLBA) SB1386 and Sarbanes-Oxley and the PaymentCard Industry (PCI) Data Security Standard and with bestpractices including SANS and OWASP in addition to manyother regulatory standards
ClickToSecure is a service that allows developers to make useof Software as a Service (SaaS) Tests are conducted remotely by
Cenzicrsquos own security expertsmdashand that means that your devel-opment team can build secure software even if it doesnrsquot haveexpertise in that area ClickToSecure accesses applicationsusing a combination of Hailstorm technology and the CenzicIntelligent Analysis (CIA) Research Lab to run assessments
In fact that CIA Lab is the foundation of the SmartAttackObjects Library Cenzic provides continuous updates based onnew vulnerabilities through the lab similar to an anti-virusmodel to help you stay ahead of the exploits and attacks Youcan use the library just as it is plus you can use CenzicrsquosSmartAttack Objects as templates that your own security archi-tects can customize for your organizationrsquos special require-ments if necessary
In addition to its software applications and services Cenzicoffers security training courses that give customers the knowl-edge and skills to use and maintain its products successfullyThe company also offers consulting services that include on-site assessment methodology and software-engineering andconsulting implementation with professional vulnerabilityconsultants who have experience in penetration testing andethical hacking
Among Cenzicrsquos many customers are Boston College DebtExchange IRIS Link and K2 Networks Boston Collegersquos devel-opment group uses Hailstorm in-house throughout its infor-mation security group to test all university Web applicationsAs they find security weaknesses developers use Hailstormreports to remediate those vulnerabilities The same group alsoaddresses its regulatory compliance issues with Hailstorm
In the case of Debt Exchange IRIS Link and K2 NetworksClickToSecure is put to work to perform security assessmentsCustomers call or fax requests to test their applications andCenzic experts test them remotely using Hailstorm in collabo-ration with the CIA Labrsquos expertise Detailed results are then
presented to the customeralong with detailed remedi-ation information
Weinschenk says thatCenzic solutions are a perfect fit for development managersand other executives Using a dashboard customers can viewapplications in the testing phase as well as the number andtypes of vulnerabilities that are found
Weinschenk adds that testing is extremely importantbecause susceptibility to intrusion can result in major recoverycosts and regulatory penalties He also reports that Cenzic hasnt tested a single application that was not vulnerable Butdonrsquot worrymdashCenzic finds the vulnerabilities before the badguys do Put Cenzic to work for you ampamp
Keeping the Bad Guys At Bay With Cenzic Solutions
1 october 2006 the artampamp science of software testing u 21
w e b s e c u r i t y
John Weinschenk
President and Chief Executive Officer
gold sponsor
Donrsquot MissOut
On Another Issue of The
Test amp QAReport
e-newsletter
Each FREE weekly issue includes original articles that interview top thought leaders in software testingand quality trends best practices
and testQA methodologiesGet must-read articles that appear
only in this e-newsletter
Sign up atwwwstpmagcomtqa
Many tools exist that help QA departments and programmerstest applications However itrsquos also crucial that senior-
level software managers have the necessary information to inter-act with executive management and accurately assess the effec-tiveness of their development teams ldquoIf addressing software qual-ity early in the development cycle is an important priority for yourcompany Stelligent can helprdquo says Burke Cox Stelligentrsquos CEO
ldquoAs the thought leaders in early testing we use commercialand open-source technologies to assess and manage softwarequality during the development and assembly of applicationsrdquoCox says ldquoThe Quality Risk Index provides an objective meas-ure of software quality that enables an entire organization toassess quality and progressrdquo
Cox believes that Stelligent (wwwstelligentcom) transformsthe way software is developed and tested By introducing com-prehensive inspection as part of a continuous process both thedevelopment team and senior management gain crucial visibil-ity into the quality risks associated with software projects This
real-time feedback lets organ-izations manage quality longbefore products are everdelivered to the QA team forevaluation
Stelligentrsquos services suchas its Kickstart Quality RiskAssessment are used byorganizations developingsoftware using managedlanguages like C orJava providing bothstatic and dynamicanalysis on the appli-cation source andcompiled code
How effective isthe Kickstart Quality
Risk Assessment Wellone customer recentlyacquired a software
product that it believedwas of high quality based on a
due-diligence assessment of the source code andend user experience However when trying to integrate the prod-uct into its larger solution it found the product had poor toler-ance to change
Thatrsquos when the customer called Stelligent whose KickstartQuality Risk Assessment identified the problems Then its
Quality Roadmap and Seven-Point Implementation Plan identified top priorities for making programmatic changes through refactoring and introduced developer testing as an active part of the continuous integration process The customer
found that Stelligentrsquos QualityRoadmap quickly solved theproblem and then its Con-tinuous Quality program kept
them solved Case closedldquoFor many organizations our Quality Risk Index is the first
objective measure of their software quality that they have everseenrdquo Cox says ldquoThis helps the software manager by providingan independent assessment of what their quality practices cur-rently achieve When working with the executive managementteam in developing schedules approving budgets and other prod-uct management activities the software manager uses Stelligentreports as a basis for discussion For example high code com-plexity with poor test coverage might make a compelling argu-ment to delay release as opposed to simply stating that the prod-uct is just not ready for prime timerdquo
Stelligentrsquos services provide a critical and objective meas-urement of your softwarersquos quality Use this intelligence tomake it difficult to enter new defects into the source repos-itory helping your developers leverage the continuous qual-ity feedback before code even reaches the QA department
Stelligent is a subsidiary of JNetDirect which makesConvergence a solution for providing real-time visibility intosoftware quality CoView which helps your team develop accu-rate and effective JUnit tests and also high-performance JDBCdrivers But long before JNetDirect acquired Stelligent it was acustomer using the Kickstart Quality Risk Assessment servicesto help build its own products Stelligentrsquos impact was so dra-matic that JNetDirect immediately saw the value of bringingStelligentrsquos services to a broader audience
ldquoDefects will always enter the build systemrdquo Cox says ldquoOurbusiness is ensuring that they cannot live there for very longQuality may not have a material impact on initial product rev-enue but it is the most significant driver of product profitabil-ity Most companies struggle with describing quality it is mixedwith subjective measures and anecdotal evidence Our customershave a quality score they can point to when setting goals andbenchmarks For organizations interested in increasing earningslowering the costs associated with poor product quality is thebest place to startrdquo
Stelligentrsquos services start with the Kickstart Quality RiskAssessmentmdashask the company how its objective measures canhelp you too improve your software quality ampamp
Stelligent Brings ObjectivityTo Quality Measurement
t e s t i n g s e r v i c e s
Burke Cox
Chief Executive Officer
1 october 2006 the artampamp science of software testing u 23
silver sponsor
The faster you find a security problem in the application lifecycle the better If developers catch vulnerabilities early they
can be fixed prior to pushing apps off to QA and customersFixing problems after they slip into a product can cost far morethan catching them early And of course if you donrsquot find the
defects someone else willldquoCenzic solutions provide
tremendous efficiencies andan immediate return on investmentrdquo says JohnWeinschenk Cenzicrsquospresident and CEO (wwwcenziccom) ldquoBy auto-mating the security test-ing process our cus-tomers can secure theirapplications faster andless expensively Likeperformance andfunctionality testingsecurity testingshould be automat-ed Vulnerabilitiesare an open invita-
tion to hackers AllWeb applications andinfrastructure can be
tested with Cenzic solutionsto locate security problemsrdquo
Cenzicrsquos two key security products are Cenzic Hailstorm andCenzic ClickToSecure These solutions provide the most com-prehensive and accurate tests in the industry Weinschenk saysthat while most vendors use scanning technology to focus onlyon commonly known vulnerabilities Hailstorm uses Cenzicrsquospatent-pending Stateful Assessment technology to automatepenetration testing by using a series of transactions to identifyvulnerabilities
Plus Hailstormrsquos SmartAttack Objects Library takes thesoftware beyond merely finding security holesmdashit can also helpenforce internal policies as well as bring organizations intoregulatory compliance with rules like the Gramm-Leach-BlileyAct (GLBA) SB1386 and Sarbanes-Oxley and the PaymentCard Industry (PCI) Data Security Standard and with bestpractices including SANS and OWASP in addition to manyother regulatory standards
ClickToSecure is a service that allows developers to make useof Software as a Service (SaaS) Tests are conducted remotely by
Cenzicrsquos own security expertsmdashand that means that your devel-opment team can build secure software even if it doesnrsquot haveexpertise in that area ClickToSecure accesses applicationsusing a combination of Hailstorm technology and the CenzicIntelligent Analysis (CIA) Research Lab to run assessments
In fact that CIA Lab is the foundation of the SmartAttackObjects Library Cenzic provides continuous updates based onnew vulnerabilities through the lab similar to an anti-virusmodel to help you stay ahead of the exploits and attacks Youcan use the library just as it is plus you can use CenzicrsquosSmartAttack Objects as templates that your own security archi-tects can customize for your organizationrsquos special require-ments if necessary
In addition to its software applications and services Cenzicoffers security training courses that give customers the knowl-edge and skills to use and maintain its products successfullyThe company also offers consulting services that include on-site assessment methodology and software-engineering andconsulting implementation with professional vulnerabilityconsultants who have experience in penetration testing andethical hacking
Among Cenzicrsquos many customers are Boston College DebtExchange IRIS Link and K2 Networks Boston Collegersquos devel-opment group uses Hailstorm in-house throughout its infor-mation security group to test all university Web applicationsAs they find security weaknesses developers use Hailstormreports to remediate those vulnerabilities The same group alsoaddresses its regulatory compliance issues with Hailstorm
In the case of Debt Exchange IRIS Link and K2 NetworksClickToSecure is put to work to perform security assessmentsCustomers call or fax requests to test their applications andCenzic experts test them remotely using Hailstorm in collabo-ration with the CIA Labrsquos expertise Detailed results are then
presented to the customeralong with detailed remedi-ation information
Weinschenk says thatCenzic solutions are a perfect fit for development managersand other executives Using a dashboard customers can viewapplications in the testing phase as well as the number andtypes of vulnerabilities that are found
Weinschenk adds that testing is extremely importantbecause susceptibility to intrusion can result in major recoverycosts and regulatory penalties He also reports that Cenzic hasnt tested a single application that was not vulnerable Butdonrsquot worrymdashCenzic finds the vulnerabilities before the badguys do Put Cenzic to work for you ampamp
Keeping the Bad Guys At Bay With Cenzic Solutions
1 october 2006 the artampamp science of software testing u 21
w e b s e c u r i t y
John Weinschenk
President and Chief Executive Officer
gold sponsor
Donrsquot MissOut
On Another Issue of The
Test amp QAReport
e-newsletter
Each FREE weekly issue includes original articles that interview top thought leaders in software testingand quality trends best practices
and testQA methodologiesGet must-read articles that appear
only in this e-newsletter
Sign up atwwwstpmagcomtqa
Many tools exist that help QA departments and programmerstest applications However itrsquos also crucial that senior-
level software managers have the necessary information to inter-act with executive management and accurately assess the effec-tiveness of their development teams ldquoIf addressing software qual-ity early in the development cycle is an important priority for yourcompany Stelligent can helprdquo says Burke Cox Stelligentrsquos CEO
ldquoAs the thought leaders in early testing we use commercialand open-source technologies to assess and manage softwarequality during the development and assembly of applicationsrdquoCox says ldquoThe Quality Risk Index provides an objective meas-ure of software quality that enables an entire organization toassess quality and progressrdquo
Cox believes that Stelligent (wwwstelligentcom) transformsthe way software is developed and tested By introducing com-prehensive inspection as part of a continuous process both thedevelopment team and senior management gain crucial visibil-ity into the quality risks associated with software projects This
real-time feedback lets organ-izations manage quality longbefore products are everdelivered to the QA team forevaluation
Stelligentrsquos services suchas its Kickstart Quality RiskAssessment are used byorganizations developingsoftware using managedlanguages like C orJava providing bothstatic and dynamicanalysis on the appli-cation source andcompiled code
How effective isthe Kickstart Quality
Risk Assessment Wellone customer recentlyacquired a software
product that it believedwas of high quality based on a
due-diligence assessment of the source code andend user experience However when trying to integrate the prod-uct into its larger solution it found the product had poor toler-ance to change
Thatrsquos when the customer called Stelligent whose KickstartQuality Risk Assessment identified the problems Then its
Quality Roadmap and Seven-Point Implementation Plan identified top priorities for making programmatic changes through refactoring and introduced developer testing as an active part of the continuous integration process The customer
found that Stelligentrsquos QualityRoadmap quickly solved theproblem and then its Con-tinuous Quality program kept
them solved Case closedldquoFor many organizations our Quality Risk Index is the first
objective measure of their software quality that they have everseenrdquo Cox says ldquoThis helps the software manager by providingan independent assessment of what their quality practices cur-rently achieve When working with the executive managementteam in developing schedules approving budgets and other prod-uct management activities the software manager uses Stelligentreports as a basis for discussion For example high code com-plexity with poor test coverage might make a compelling argu-ment to delay release as opposed to simply stating that the prod-uct is just not ready for prime timerdquo
Stelligentrsquos services provide a critical and objective meas-urement of your softwarersquos quality Use this intelligence tomake it difficult to enter new defects into the source repos-itory helping your developers leverage the continuous qual-ity feedback before code even reaches the QA department
Stelligent is a subsidiary of JNetDirect which makesConvergence a solution for providing real-time visibility intosoftware quality CoView which helps your team develop accu-rate and effective JUnit tests and also high-performance JDBCdrivers But long before JNetDirect acquired Stelligent it was acustomer using the Kickstart Quality Risk Assessment servicesto help build its own products Stelligentrsquos impact was so dra-matic that JNetDirect immediately saw the value of bringingStelligentrsquos services to a broader audience
ldquoDefects will always enter the build systemrdquo Cox says ldquoOurbusiness is ensuring that they cannot live there for very longQuality may not have a material impact on initial product rev-enue but it is the most significant driver of product profitabil-ity Most companies struggle with describing quality it is mixedwith subjective measures and anecdotal evidence Our customershave a quality score they can point to when setting goals andbenchmarks For organizations interested in increasing earningslowering the costs associated with poor product quality is thebest place to startrdquo
Stelligentrsquos services start with the Kickstart Quality RiskAssessmentmdashask the company how its objective measures canhelp you too improve your software quality ampamp
Stelligent Brings ObjectivityTo Quality Measurement
t e s t i n g s e r v i c e s
Burke Cox
Chief Executive Officer
1 october 2006 the artampamp science of software testing u 23
silver sponsor
Donrsquot MissOut
On Another Issue of The
Test amp QAReport
e-newsletter
Each FREE weekly issue includes original articles that interview top thought leaders in software testingand quality trends best practices
and testQA methodologiesGet must-read articles that appear
only in this e-newsletter
Sign up atwwwstpmagcomtqa
Many tools exist that help QA departments and programmerstest applications However itrsquos also crucial that senior-
level software managers have the necessary information to inter-act with executive management and accurately assess the effec-tiveness of their development teams ldquoIf addressing software qual-ity early in the development cycle is an important priority for yourcompany Stelligent can helprdquo says Burke Cox Stelligentrsquos CEO
ldquoAs the thought leaders in early testing we use commercialand open-source technologies to assess and manage softwarequality during the development and assembly of applicationsrdquoCox says ldquoThe Quality Risk Index provides an objective meas-ure of software quality that enables an entire organization toassess quality and progressrdquo
Cox believes that Stelligent (wwwstelligentcom) transformsthe way software is developed and tested By introducing com-prehensive inspection as part of a continuous process both thedevelopment team and senior management gain crucial visibil-ity into the quality risks associated with software projects This
real-time feedback lets organ-izations manage quality longbefore products are everdelivered to the QA team forevaluation
Stelligentrsquos services suchas its Kickstart Quality RiskAssessment are used byorganizations developingsoftware using managedlanguages like C orJava providing bothstatic and dynamicanalysis on the appli-cation source andcompiled code
How effective isthe Kickstart Quality
Risk Assessment Wellone customer recentlyacquired a software
product that it believedwas of high quality based on a
due-diligence assessment of the source code andend user experience However when trying to integrate the prod-uct into its larger solution it found the product had poor toler-ance to change
Thatrsquos when the customer called Stelligent whose KickstartQuality Risk Assessment identified the problems Then its
Quality Roadmap and Seven-Point Implementation Plan identified top priorities for making programmatic changes through refactoring and introduced developer testing as an active part of the continuous integration process The customer
found that Stelligentrsquos QualityRoadmap quickly solved theproblem and then its Con-tinuous Quality program kept
them solved Case closedldquoFor many organizations our Quality Risk Index is the first
objective measure of their software quality that they have everseenrdquo Cox says ldquoThis helps the software manager by providingan independent assessment of what their quality practices cur-rently achieve When working with the executive managementteam in developing schedules approving budgets and other prod-uct management activities the software manager uses Stelligentreports as a basis for discussion For example high code com-plexity with poor test coverage might make a compelling argu-ment to delay release as opposed to simply stating that the prod-uct is just not ready for prime timerdquo
Stelligentrsquos services provide a critical and objective meas-urement of your softwarersquos quality Use this intelligence tomake it difficult to enter new defects into the source repos-itory helping your developers leverage the continuous qual-ity feedback before code even reaches the QA department
Stelligent is a subsidiary of JNetDirect which makesConvergence a solution for providing real-time visibility intosoftware quality CoView which helps your team develop accu-rate and effective JUnit tests and also high-performance JDBCdrivers But long before JNetDirect acquired Stelligent it was acustomer using the Kickstart Quality Risk Assessment servicesto help build its own products Stelligentrsquos impact was so dra-matic that JNetDirect immediately saw the value of bringingStelligentrsquos services to a broader audience
ldquoDefects will always enter the build systemrdquo Cox says ldquoOurbusiness is ensuring that they cannot live there for very longQuality may not have a material impact on initial product rev-enue but it is the most significant driver of product profitabil-ity Most companies struggle with describing quality it is mixedwith subjective measures and anecdotal evidence Our customershave a quality score they can point to when setting goals andbenchmarks For organizations interested in increasing earningslowering the costs associated with poor product quality is thebest place to startrdquo
Stelligentrsquos services start with the Kickstart Quality RiskAssessmentmdashask the company how its objective measures canhelp you too improve your software quality ampamp
Stelligent Brings ObjectivityTo Quality Measurement
t e s t i n g s e r v i c e s
Burke Cox
Chief Executive Officer
1 october 2006 the artampamp science of software testing u 23
silver sponsor
Many tools exist that help QA departments and programmerstest applications However itrsquos also crucial that senior-
level software managers have the necessary information to inter-act with executive management and accurately assess the effec-tiveness of their development teams ldquoIf addressing software qual-ity early in the development cycle is an important priority for yourcompany Stelligent can helprdquo says Burke Cox Stelligentrsquos CEO
ldquoAs the thought leaders in early testing we use commercialand open-source technologies to assess and manage softwarequality during the development and assembly of applicationsrdquoCox says ldquoThe Quality Risk Index provides an objective meas-ure of software quality that enables an entire organization toassess quality and progressrdquo
Cox believes that Stelligent (wwwstelligentcom) transformsthe way software is developed and tested By introducing com-prehensive inspection as part of a continuous process both thedevelopment team and senior management gain crucial visibil-ity into the quality risks associated with software projects This
real-time feedback lets organ-izations manage quality longbefore products are everdelivered to the QA team forevaluation
Stelligentrsquos services suchas its Kickstart Quality RiskAssessment are used byorganizations developingsoftware using managedlanguages like C orJava providing bothstatic and dynamicanalysis on the appli-cation source andcompiled code
How effective isthe Kickstart Quality
Risk Assessment Wellone customer recentlyacquired a software
product that it believedwas of high quality based on a
due-diligence assessment of the source code andend user experience However when trying to integrate the prod-uct into its larger solution it found the product had poor toler-ance to change
Thatrsquos when the customer called Stelligent whose KickstartQuality Risk Assessment identified the problems Then its
Quality Roadmap and Seven-Point Implementation Plan identified top priorities for making programmatic changes through refactoring and introduced developer testing as an active part of the continuous integration process The customer
found that Stelligentrsquos QualityRoadmap quickly solved theproblem and then its Con-tinuous Quality program kept
them solved Case closedldquoFor many organizations our Quality Risk Index is the first
objective measure of their software quality that they have everseenrdquo Cox says ldquoThis helps the software manager by providingan independent assessment of what their quality practices cur-rently achieve When working with the executive managementteam in developing schedules approving budgets and other prod-uct management activities the software manager uses Stelligentreports as a basis for discussion For example high code com-plexity with poor test coverage might make a compelling argu-ment to delay release as opposed to simply stating that the prod-uct is just not ready for prime timerdquo
Stelligentrsquos services provide a critical and objective meas-urement of your softwarersquos quality Use this intelligence tomake it difficult to enter new defects into the source repos-itory helping your developers leverage the continuous qual-ity feedback before code even reaches the QA department
Stelligent is a subsidiary of JNetDirect which makesConvergence a solution for providing real-time visibility intosoftware quality CoView which helps your team develop accu-rate and effective JUnit tests and also high-performance JDBCdrivers But long before JNetDirect acquired Stelligent it was acustomer using the Kickstart Quality Risk Assessment servicesto help build its own products Stelligentrsquos impact was so dra-matic that JNetDirect immediately saw the value of bringingStelligentrsquos services to a broader audience
ldquoDefects will always enter the build systemrdquo Cox says ldquoOurbusiness is ensuring that they cannot live there for very longQuality may not have a material impact on initial product rev-enue but it is the most significant driver of product profitabil-ity Most companies struggle with describing quality it is mixedwith subjective measures and anecdotal evidence Our customershave a quality score they can point to when setting goals andbenchmarks For organizations interested in increasing earningslowering the costs associated with poor product quality is thebest place to startrdquo
Stelligentrsquos services start with the Kickstart Quality RiskAssessmentmdashask the company how its objective measures canhelp you too improve your software quality ampamp
Stelligent Brings ObjectivityTo Quality Measurement
t e s t i n g s e r v i c e s
Burke Cox
Chief Executive Officer
1 october 2006 the artampamp science of software testing u 23
silver sponsor