Presenter: Byron Goodman National Director of Quality Assurance Neudesic.

• Slide 1

Presenter: Byron Goodman National Director of Quality Assurance Neudesic Slide 2 Some thoughts on automation Successful test automation Components of a framework Some coding standards Standard routines Slide 3 Let the machines do the tests Easy to create alternate scenarios by leveraging existing tests Using tools can create more interest in testing activities Potential to reduce testing cycle time Effective at catching regression bugs Slide 4 Tests break too often! Pass or fail results may not be reliable Tools are expensive, although there are some good open source solutions Tools may oversimplify or overcomplicate the degree of effort required to test applications Slide 5 More time spent working on automated tests than time needed to do the manual testing Sporadic interest in implementing automated testing Statements by others that automated testing is: a sham, ineffective, waste of time Shelving of test automation in the interest of project success Slide 6 As with all project related work, the time spent creating automated testing must offset this effort by the return of value. This is demonstrated in various ways: Less time expended during the testing cycle than if manual testing is the only solution More thorough and consistent regression coverage Reduce human error and boredom Frees up people to test new functionality while computers test existing functionality Slide 7 Strategy start simple and work towards a more complete and complex solution Timing start using tests as soon as they are written, Do not wait for a completed suite Buy-in Demonstrate to others that each test does exactly what it is intended to so that there is confidence in results Traceability Each automated test must trace back to a specific automated test Slide 8 Generally speaking, some automated testing is better than no automated testing Select the most simple of tests to automate Begin by using record and playback to minimize time investment but to show that the tool does prove to be effective Slide 9 For years, I have insisted that no tests will be created using record and playback I will admit that there is a place for R&P Very light use to prove out the tool and to minimize the reluctance people may have to use complicated technology However, R&P is does not have much gain in efficiencies over manual testing Slide 10 Many of us have experience with testing frameworks Other names DLLs, namespace, test library Its just a fancy name for code that is used to automate tests A testing framework is a set of routines that are reused by tests that will speed up test development while increasing test reliability Slide 11 Needs to be recognized to have the same rigor as any development effort Adds value to the overall test effort Implements complex and redundant routines for tests to leverage Minimizes the impact of a changing application to the tests, which reduces test maintenance Slide 12 Each layer is developed to promote portability and reusability The more generic a layer, the more reusable it needs to be i.e. the Generic Routines should be able to work for everyone all the time regardless of the environments and AUTs Slide 13 Slide 14 Common routines that do not interact with any application Generic UI layer, non-technology-specific manipulation routines Technology-specific layer routines Application-specific layer routines Application-specific Scenario/transactional layer routines Tests layer Slide 15 Not related to the application or an interface Build more capability and utility that the test tool may not have Functionality that a tester may need for any reason If written properly, these routines will work with test tools that use the same programming language Slide 16 Some control types, like grids, that are generic and rather independent from the technology that they are written in This is a good place to create additional or alternate logging Slide 17 For handling specific technologies i.e. Browsers, WinForms and WPF. Custom controls will be wrapped with calls that are identical to similar standard controls Create interfaces for new versions of technology if the tool vendor does not yet have the new technology versions integrated Slide 18 Routines that are specific to the AUT Repository for UI mappings Defines a handler for every individual form/page AUT service interfaces defined and wrapped Serves the calls needed for tests to interact with the AUT Slide 19 Use this as a distinguishable layer if the routines for scenarios are complex and extensive May include multiple forms/pages In cases where there will not be a lot of routines, this can be combined into the Application-specific manipulation routines Slide 20 This layer will contain only tests No test calls any other test Tests call into the other layers Tests should be very simple as all the complexity is implemented in the other layers Slide 21 Slide 22 Using consistent coding standards promoted reusability and maintainability Standards need to be documented and followed similar to what project development would do Use code reviews to enforce standards Slide 23 Naming conventions for variables and methods/functions Use human readable names! Limit code complexity Separation of layers Comments Slide 24 Do not use the control type in the name, Address, not AddressEdit No duplication of control names for any specific window if the tools would normally allow this Consistent naming of similar controls form to form, even if the forms label the controls differently Slide 25 Execute anywhere tests need to written so that they do not rely upon a very specific environment configuration, unless that is the purpose of the test Unattended testing tests that need human interaction to execute should be segregated so that the other tests can all be run unattended Slide 26 No hard coding use configuration files or similar mechanisms to store information or environment values In tests, avoid using the automation tool syntax Tests should call into the other layers This will make tests less likely to fail if the tool vendor modifies their syntax Can make it easier to port tests to new automation tools Slide 27 Overuse logging Design for distributed testing and for concurrent execution of tests An automated test will test just a single test No test ever depends upon the successful actions or completion of a previous test Computers dont care if they do repetitive actions Slide 28 Baseline (Initialization) Navigation Editing Verification Tools already have their own routines defined for some of this however, you will get more flexibility and maintainability using these patterns Slide 29 Will set the environment to the proper state to start a test This state is defined for each AUT and test team Can be simple to very complex Determined by defining a known state that all tests can reliably execute and get consistent results Slide 30 Used to determine if a page/form is active and ready for interaction Will navigate to the desired page/form if the steps required are not complex, or the steps will not alter the outcome of any test Every page/form will have an unique navigation routines created Slide 31 Enters data into fields, pushes buttons, sets checkboxes, selects values in lists, etc. The control types should not matter when calling this routine it will figure out what the control type is during test execution and interact with the controls appropriately Defined as an unique routine for every individual page/form Slide 32 Reads data from all controls types Determines if the values read result in a pass or fail Responsible for logging results Defined as an unique routine for every individual page/form Slide 33 Solution layout Actual code examples Dealing with particularly difficult controls Configuration management Skill sets Specific tools capabilities Slide 34 Beginning automated tests Test framework goals Layers of a framework Coding standards Initial routines that every page/form requires Slide 35 Thanks for your participation [email protected]