Test Automation Patterns: Issues and Solutions

TG Half-day Tutorials

5/6/2014 8:30:00 AM

Test Automation Patterns:

Issues and Solutions

Presented by:

Seretta Gamba

Dorothy Graham

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073

888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com

Seretta Gamba Steria Mummert ISS GmbH

Seretta Gamba has more than thirty years of experience in software development. As test manager at Steria Mummert ISS GmbH, Seretta was charged with improving the test automation process. After studying the current strategies, she developed a style of keyword-driven testing and a framework to support it. In 2009, the framework was extended to support manual testing. Speaking about this at EuroSTAR, Seretta got the attention of Dorothy Graham who subsequently invited her to contribute a chapter to Dot’s bookExperiences of Test Automation. After reading the entire book, Seretta noticed recurring patterns in solving automation problems and is currently intent on cataloguing test automation patterns.

Dorothy Graham Independent Test Consultant

In software testing for forty years, Dorothy Graham is coauthor of four books—Software Inspection, Software Test Automation, Foundations of Software Testing and Experiences of Test Automation—and is currently working with Seretta Gamba on a new book on test automation patterns. A popular and entertaining speaker at conferences and seminars worldwide, Dot has attended STAR conferences since the first one in 1992. She was a founding member of the ISEB Software Testing Board and a member of the working party that developed the ISTQB Foundation Syllabus. Dot was awarded the European Excellence Award in Software Testing in 1999 and the first ISTQB Excellence Award in 2012. Learn more about Dot at DorothyGraham.co.uk.

[email protected] [email protected]

© Seretta Gamba and Dorothy Graham 2014 www.DorothyGraham.co.uk

1

Patterns in Test Automation:

Issues and Solutions Prepared and presented by

Seretta Gamba


© Seretta Gamba, Dorothy Graham 2014

Dorothy Graham

@DorothyGraham [email protected] www.DorothyGraham.co.uk

2

Tutorial description •  Testers often encounter problems when automating system

level test execution. These problems often have known solutions, yet many testers are not aware of them.

•  Recognizing the commonality of these test automation issues and their solutions, Seretta Gamba and Dorothy Graham have organized them into a set of Test Automation Patterns.

•  A pattern is a general, reusable solution to a commonly occurring problem, commonly used in development.

•  Issues include NO PREVIOUS AUTOMATION, HIGH ROI EXPECTATIONS, HIGH TEST MAINTENANCE COST

•  Patterns include MAINTAINABLE TESTWARE, TOOL INDEPENDENCE, MANAGEMENT SUPPORT



3

Tutorial objectives •  help you recognize test automation issues

–  share information with others having similar problems –  look in detail at a couple of selected common issues

•  explore ways of solving automation issues using patterns –  look at alternative solutions, depending on context –  ways to implement selected patterns

•  work on your own issues* –  identify your most pressing issues –  investigate patterns that may help address them

•  NOTE: system level automation, not unit level •  NOTE: test automation patterns, not test patterns

*using [offline] Test Automation Patterns wiki on your laptop

4

Issues (problems)

•  what problems affect you in test automation? – what causes the most trouble? – what takes longer than it should? – what is holding you back in your automation? – what are you struggling with? – what would you most like to improve?



5

Practical work •  form groups of people with similar problems:

– not done automation before – want to improve/revive automation, but suffer from:

•  lack of support •  lack of resources •  lack of direction •  lack of specific knowledge •  expectations for automation not met •  unsatisfactory quality of automation

•  fill out page 1 of your worksheet –  introduce yourselves in your group

6

Contents Background

Issues and Patterns Examples: issues to patterns

Diagnostic process A case study

The Wiki Conclusion

Test Automation Patterns



7

Origin of patterns in software – 1970s: architect Christopher Alexander describes

patterns for buildings (e.g. light from 2 walls) – 1987: Kent Beck & Ward Cunningham

experimented & presented at OOPSLA – 1994: “Gang of Four” wrote a book applying

pattern principles to software design •  “Design Patterns: elements of reusable object-oriented

software”, Erich Gamma, Richard Helm, Ralph Johnson, John Vlissides

– now widely accepted in software development community

8

What is a pattern? – a generally reusable solution to a commonly

occurring problem within a given context – a description or template for how to solve a

problem that can be used in many different situations

– a pattern is NOT •  a finished solution that can be ‘plugged in’ or

transformed directly into code •  prescriptive •  a set of steps

– a pattern needs to be implemented within context



9

Where did these Test Automation Patterns come from?

– Experiences book published 2012 – Seretta contributed Ch 21

•  Automation through the back door, by supporting manual testing

– she is a developer, used to using patterns

– when she read the other chapters, she thought: “Patterns! Patterns!”

– she wrote up test automation patterns as a book & asked Dot to join her

– Word didn’t work very well, so we made a wiki

10

Contents Background



The Wiki Conclusion




11

Test Automation Issues

–  An Issue is a problem that testers and/or test automators encounter when trying to do test automation.

–  They are things that take longer than they should, that hold you back in what you want to automate, or that are just too much trouble

12

Test Automation Issues (~50) –  Process Issues: problems that occur when the test

automation process has not yet reached the necessary maturity

–  Management Issues: problems that occur when management has not given the necessary support (budget, resources, team).

–  Design Issues: problems that occur when an efficient testware architecture and maintainability are not built in from the very beginning.

–  Execution Issues: problems that occur when the automated tests are run (unpredictable results etc.).



13

Test Automation Patterns •  General reusable solution to a commonly occurring

problem within a given context. •  a way of solving an issue or problems in test

automation that has worked in practice for many people

•  a pattern is not prescriptive or a set of steps, but rather ideas to consider to see if they would apply in your situation or context

Pattern expert knowledge proven by repeated experience

14

Test Automation Patterns (~70) •  Process Patterns: show how the test automation process

should be set up or how it can be improved. •  Management Patterns: show how to manage test

automation both as an autonomous project or integrated in the development process

•  Design Patterns: show how to design the test automation testware so that it will be efficient and easy to maintain

•  Execution Patterns: show how to take care that test execution is easy and reliable.



15

Contents Background



The Wiki Conclusion


16

Test automation Issues: 4 examples •  HIGH MAINTENANCE COST

–  Maintenance of the test automation scripts takes a long time and costs more than it’s worth.

•  NO PREVIOUS TEST AUTOMATION –  Your first time automating, or automation has failed and

you are starting again. •  STALLED AUTOMATION

–  Automation has been tried, but it never “got off the ground” or has now fallen into disuse.

•  UNREALISTIC EXPECTATIONS –  Management has unrealistic expectations about what

test automation can & can’t do.



17

Issue: HIGH MAINTENANCE COST •  Maintenance of the test automation scripts

takes a long time and costs more than it’s worth

•  Questions: – what cost (or time) would be acceptable? – how is the cost measured? – Note: answers may be different in different

contexts

18

Issue: HIGH MAINTENANCE COST 1 •  What are your symptoms?

– When software changes, e.g. window sequence is different, tests fail. More effort to edit the existing tests than to start again. (Capture/replay used)

– Are the scripts structured and reusable? •  1) No, scripts are mainly “stand-alone” without much

reuse, automators are “re-inventing” •  2) Yes, structured and reusable scripts, but it needs

automators to add new tests, and we are short of time and/or automators

visit offline wiki



19

Issue: HIGH MAINTENANCE COST 2 •  What are your symptoms?

– No time allowed to maintain the automated tests

– Tests are written in the tool’s language, but the tool has changed, or we have to change tools

20

HIGH MAINTENANCE COST (Tour) - 1

•  Symptom: SUT changes -> large cost to update – MAINTAINABLE TESTWARE – Symptom: Scripts are mainly “stand-alone” without

much reuse, automators are re-inventing •  GOOD PROGRAMMING PRACTICES

– DESIGN FOR REUSE, OBJECT MAP

– Symptom: Structured & reusable scripts, automator / time shortage •  DOMAIN-DRIVEN TESTING

–  ABSTRACTION LEVELS, KEYWORD-DRIVEN TESTING



21

HIGH MAINTENANCE COST (Tour) - 2 •  Symptom: No time to maintain automated

tests (we know we should) – MANAGEMENT SUPPORT

•  Explain technical debt, SELL THE BENEFITS

•  Symptom: Scripts in tool language, tool changed or having to change tools – TOOL INDEPENDENCE – ABSTRACTION LEVELS

22

Patterns for an example Issue •  Issue: HIGH MAINTENANCE COST •  resolving patterns:

– ABSTRACTION LEVELS – MAINTAINABLE TESTWARE – MODEL-BASED TESTING – MANAGEMENT SUPPORT – TOOL INDEPENDENCE – WHOLE TEAM APPROACH

•  one issue leads to many patterns –  the best one(s) for you, at this time, depends on

your context and situation

More about some of these

patterns



23

Abstraction levels: Testware architecture

abstraction here: easier to write automated tests widely used

abstraction here: easier to maintain, and change tools long life

testware archite

cture

Testers

Test Execu/on Tool runs scripts

High Level Keywords

Structured Scripts

structured testware

Test Automator(s)

write tests (in DSTL)

24

re-usable modules / script library

Progression of automation implementation

low-level instructions

structured code

data driven

keyword/ domain driven

manual test scripts

input data

test definitions

get quote - motor, age, … create policy – motor, age, …

Technical Test Analyst Test Analyst

easier maintenance Source: Mark Fewster, Grove Consultants (grove.co.uk)



25

Data file: TestCase2

FILE ADD MOVE DELETE Europe France Germany 1,3 2,2 1 5,3

Data-driven example

Data file: TestCase1

FILE ADD MOVE DELETE countries Sweden USA 4,1 Norway 2 7

For each TESTCASE OpenDataFile(TESTCASEn) ReadDataFile(RECORD) For each record ReadDataFile(RECORD) Case (Column(RECORD)) FILE: OpenFile(INPUTFILE) ADD: AddItem(ITEM) MOVE: MoveItem(FROM, TO) DELETE: DeleteItem(ITEM) ….. Next record Next TESTCASE Control script

26

Comparison of data- and keyword-driven

ScribbleOpen Europe AddToList France Italy MoveItem 1 to 3 MoveItem 2 to 2 DeleteItem 1 MoveItem 5 to 2 SaveAs Test2

keyword approach data-driven approach

FILE ADD MOVE DELETE SAVE Europe France Italy 1,3 2,2 1 5,2 Test2

which is easier to read/understand?

what happens when the test becomes large and complex?

this looks more like a test



27

Contents Background



The Wiki Conclusion


28

What is the diagnostic process?

–  We want to identify the most pressing automation problems (issues) by asking specific questions (just like a doctor would do)

–  If the issue is known, it can be solved with the pattern approach

–  Afterwards you can address the next most pressing problem



29

Why is diagnosis helpful?

–  You don‘t have to study all the issues and patterns (50+70)

–  The questions lead you to the causes of your problems

–  You can focus on the most effective patterns –  You can repeat the process as often as you

want

30

What do you do when you feel sick?

•  Wait until it’s over

•  Search on the internet for a cure

•  Ask a friend for advice

•  Study medicine

•  Go to your doctor



31

What does the doctor do?

•  Asks you why you came

•  Measures blood pressure etc.

•  Asks more detailed questions

•  Tells you what you have

•  Orders some medicine

32

Why go to the doctor?

You take the medicine

By asking good questions the doctor diagnosed your illness and could give you the appropriate medicine: You get better!



33

What do you do if your automation is sick?

•  Just go on as always •  Search on the internet for a cure •  Pay a consultant •  Become a test automation guru •  Come to this conference to the automation doctor

34

What does the Automation doctor do?

•  Asks you why you came

•  Does not measure blood pressure etc.

•  Asks more detailed questions

•  Tells you what your issue(s) are

•  Suggests what pattern(s) to apply



35

Diagnostic questions

You apply the patterns

By asking good questions you can identify your issue(s) and thus select the most appropriate pattern(s): Your automation gets better!

36

3 Examples

Diagnose process



37

Example 1: First question (Q1)

□  Completely happy?

□  Want to improve or revive your test automation?

□  Never done test automation before? X

NO PREVIOUS TEST

AUTOMATION

12%

How satisfied are you with your current automation?

38

Issue description: NO PREVIOUS TEST AUTOMATION

You are supposed to start automating tests, but neither you nor your team has any experience in test automation and it hasn’t ever been implemented in the company, or it was tried earlier and failed so you need to start again

•  You have never successfully got going with test automation.

•  You have tried in the past with test automation but it was abandoned.

•  You are wanting to automate in a way that you haven't done before (e.g. GUI level system testing if you have automated unit tests already).



39

Issue solutions: NO PREVIOUS TEST AUTOMATION

Patterns: 1.  SET CLEAR GOALS 2.  MANAGEMENT SUPPORT 3.  DEDICATED RESOURCES 4.  RIGHT TOOLS 5.  AUTOMATION ROLES 6.  PLAN SUPPORT ACTIVITIES 7.  MAINTAINABLE TESTWARE 8.  AUTOMATE WHAT’S NEEDED 9.  TAKE SMALL STEPS 10. UNATTENDED TEST EXECUTION

40

Pattern: SET CLEAR GOALS

Define the automation objectives from the very beginning in a way that is clear and understandable to all.

Context This pattern is always applicable, although the goals may be different at different stages. • when you first consider test automation • when an automation initiative is getting started • when your automation is going well • when you want to revitalise a stalled automation effort.

If you don‘t know what your goals are, how do you know that you are going in the right direction?



41


Description • automation objectives defined clearly up front no disappointment later.

• Inform your managers about what is feasible and what is not.

Goals should be measurable so that you can tell if you have achieved them or not.

42


Implementation examples • Selected regression tests should run x-times faster and y-times more often. • Tests too complex to perform manually are to be automated. Define exactly which ones. • …..

Not good goals • confuse the goals for automation with goals for testing. • Automate all manual tests



43


Potential problems Be sure to remind management that it may not be possible or advisable to automate every test case.

44

Example 2: First question (Q1)

□  Are you completely happy with your automation at this time?

□  Do you want to improve or revive your test automation?

□  Have you ever done test automation before?

X

How satisfied are you with your current automation?



45

Example 2: Second question (Q1.2)

What describes the most pressing problem you have to tackle at the moment?

□  Lack of Support?

□  Lack of specific knowledge?

□  Expectations not met / Unsatisfactory quality of automation?

□  Lack of Resources?

□  Lack of Direction?

57%

X

34%

28%

41% 29%

46

Example 2: Third question (Q1.2.2)

What is the main reason for the lack of resources?

□  Expenses not budgeted □  Not enough machines □  Shared resources with testing /

development □  Not enough tool licenses □  Insufficient time planned □  Training hasn’t been planned □  Nobody is in charge

X

SCHEDULE SLIP

64%

35%



47

Issue: SCHEDULE SLIP The planned automation is not keeping up with its schedule for developing the automation or for automating tests

•  Test automation is done only in people's spare time.

•  Team members are working on concurrent tasks which take priority over automation tasks.

•  Schedules for what automation can be done were too optimistic.

•  Necessary software or hardware is not available on time or has to be shared with other projects

•  The planned schedule was not realistic

48

Issue: SCHEDULE SLIP

Most recommended: SHORT ITERATIONS WHOLE TEAM APPROACH DEDICATED RESOURCES PLAN SUPPORT ACTIVITIES

Other useful patterns: FULL TIME JOB MANAGEMENT SUPPORT



49

Pattern: DEDICATED RESOURCES

The test automation team is available full time and all the needed tools and machines are in place

Context Not applicable for very small organizations with only one or two testers.

Description • A test automation team is assigned to work on test automation full-time, without other responsibilities. • Any, tools, utilities, machines or other resources are provided as needed. • You can get the help of specialists when you need it.

50


Implementation • Test automation should be done as a FULL TIME JOB • PLAN SUPPORT ACTIVITIES • Tools and machines to run the tests need to be available, and ideally dedicated to running automated tests. GET ON THE CLOUD for cost-efficiency. • SHARE INFORMATION



51


Potential problems It is often the case that resources are severely limited, but expecting people to "do test automation in their spare time" is very unlikely to work, and is not efficient.

52

Example 3: Second question (Q1.2)

What describes the most pressing problem you have to tackle at the moment?

□  Lack of Support?

□  Lack of specific knowledge?

□  Unsatisfactory quality of automation?

□  Lack of Resources?

□  Lack of Direction?

X

29%



53

Example 3: Third question (Q1.2.5)

What do you think is the main reason for the lack of quality?

X

□  The expected Return On Investment (ROI) hasn't been achieved

□  Automated scripts don’t execute reliably

□  The automation project is not on schedule

□  Maintenance of automation is too expensive

□  ……

50%

44%

54

Example 3: Fourth question (Q1.2.5.4)

What makes maintenance costs too high?

X

□  Adapting the automation scripts to the current version of the SUT

□  Refactoring of the automated scripts □  Changing tools □  Creation or recovery of test data



55

Example 3: Fifth question (Q1.2.5.4.4)

What is the data issue that currently gives you the most headaches?

X □  Setting up the initial environments □  Preparing the expected results □  Reusing existing data □  Disposing of redundant data

We are getting more and more specific

56

Example 3: Sixth question (Q1.2.5.4.4.1)

What is so difficult in setting up the initial environments?

□  Insufficient setup information □  Complex environment □  Variable database contents (for

instance anonymized customer data) □  Tests run on multiple platforms

X



57

Example 3: Seventh question (Q1.2.5.4.4.1.1)

Why is there insufficient information in the test cases about the set-up of the test?

□  Testers are not very experienced □  Testers know the system so well that

they don't need much description in the test cases, but don't have time to explain them to you.

□  The Software Under Test (SUT) is still very unstable and the requirements change daily

X

INADEQUATE SUPPORT

58

Issue: INADEQUATE SUPPORT The test automation team doesn't get adequate support from management, testers or other specialists

•  Management knows too little about benefits or needed investments

•  Testers don’t have time to support the test automation team.

•  You need support, but nobody has time for you

•  Schedules were too optimistic. •  Necessary software or hardware is not

available on time or has to be shared with other projects



59

Issue: INADEQUATE SUPPORT

Most recommended: MANAGEMENT SUPPORT PLAN SUPPORT ACTIVITIES

Other useful patterns: SELL THE BENEFITS SHARE INFORMATION SHORT ITERATIONS

60

Contents Background



The Wiki Conclusion




61

Seretta’s story: background •  company background

– Steria-Mummert-ISS, Hamburg / Germany – Standard Software for insurances

•  my background – developer –  test manager

•  2001, wanted to start automation – NO PREVIOUS AUTOMATION

•  this is the issue we had then

62

NO PREVIOUS TEST AUTOMATION •  resolving patterns:

1.  SET CLEAR GOALS 2.  MANAGEMENT SUPPORT 3.  DEDICATED RESOURCES 4.  RIGHT TOOLS 5.  AUTOMATION ROLES 6.  PLAN SUPPORT ACTIVITIES 7.  MAINTAINABLE TESTWARE 8.  AUTOMATE WHAT’S NEEDED 9.  TAKE SMALL STEPS 10. UNATTENDED TEST EXECUTION

automate regression tests for the

registration product

We chose an older tool that could drive our application reliably

I knew how to write good code for the automation

Done

Done

Done SET CLEAR GOALS RIGHT TOOLS MAINTAINABLE TESTWARE AUTOMATE WHAT’S NEEDED TAKE SMALL STEPS UNATTENDED TEST EXECUTION



63

MAINTAINABLE TESTWARE •  Summary:

Design testware so it doesn’t have to be updated for every little change in the Software Under Test (SUT)

•  Description: •  Identify the most costly and/or most frequent changes •  Design testware to cope best with those changes

•  Implementation •  Example: objects are frequently renamed: use OBJECT

MAP or devise a translation table for names •  GOOD DEVELOPMENT PROCESS and GOOD

PROGRAMMING PRACTICESS •  DOMAIN-DRIVEN TESTING using Keywords

64

Command-driven testing •  developed a keyword-type test architecture

– minimise scripts that interface directly to the tool •  TOOL INDEPENDENCE

– organised into a Driver-part and a Data-part •  DATA-DRIVEN TESTING

– developed ISS Test Station to make it all work •  TEST AUTOMATION FRAMEWORK

– keywords as simple commands • e.g. “INPUT”, “SELECT” •  KEYWORD-DRIVEN TESTING



65

66

Success •  the automation gave us good benefits

– we had a good architecture and framework – easy to add new automated tests – company-wide automation standards – automated everything we had planned – expected our ratio of automated to manual tests to

continue to grow



67

What happened next: Stagnation •  marketing success = automation problem

–  increase in business, new features – developers and testers needed on business-

critical tasks, taken off automation –  leading to new issues:

•  INADEQUATE SUPPORT •  INADEQUATE COMMUNICATION • NO INFO ON CHANGES

– no time for automation, which would have given them more time!

68

NO PREVIOUS TEST AUTOMATION •  resolving patterns used/missed:

1.  SET CLEAR GOALS 2.  MANAGEMENT SUPPORT 3.  DEDICATED RESOURCES 4.  RIGHT TOOLS 5.  AUTOMATION ROLES 6.  PLAN SUPPORT ACTIVITIES 7.  MAINTAINABLE TESTWARE 8.  AUTOMATE WHAT’S NEEDED 9.  TAKE SMALL STEPS 10. UNATTENDED TEST EXECUTION

I was doing the automation in my

“spare time”

I was automator, tester, developer and project leader

at first I didn’t need support, didn’t think about this

Initiated by management, but not enough



69

MANAGEMENT SUPPORT - 1 •  Implementation Build a convincing TEST AUTOMATION BUSINESS CASE.

Test automation can be quite expensive and requires, especially at the beginning, a lot of effort.

Without a business case there is no real commitment from management to continue to support test automation, e.g. when development projects need the same resources. Just as for testing in general, test automation doesn’t come first! Management initiation of the automation was good, but it wasn’t enough – needs ongoing support .

70

MANAGEMENT SUPPORT - 2 A good way to convince management is to DO A PILOT. In this way they can actually “touch” the advantages of test automation and it will be much easier to win them over.

•  SET CLEAR GOALS •  RIGHT TOOLS •  AUTOMATION ROLES •  TAKE SMALL STEPS

• 

•  Take time for debriefing when you are through and don't forget to LEARN FROM MISTAKES

• 

I didn’t even notice that it was missing!



71

Other Issues •  INADEQUATE TEAM

I could not select the team and had to manage with colleagues who were not needed for development!

•  AUTOMATION ROLES •  FULL TIME JOB

If you have a pattern, that at least confronts you with the problem from the beginning. You cannot say afterwards that you didn’t think about it! .

72

Case study conclusion •  we unknowingly applied some patterns

– enabled us to achieve success •  we didn’t think about other patterns

–  they caused us serious problems later on •  many other people have similar experiences

– some succeed, some don’t –  the test automation patterns can help you on the

road to success



73

Contents Background



The Wiki Conclusion


74

Introduction to the wiki

•  the Test Automation Patterns wiki – Diagnostics – questions to help identify problems –  ISSUES – common problems in automation – PATTERNS – proven ways to overcome problems

•  in this tutorial – we will use an off-line version of the wiki if no

internet access in the room – gives you a chance to get started with it – answer any questions you have – prepare you to get the most benefit from it later



75

Test automation patterns wiki •  purpose: share information, ideas and

experiences about test automation, using issues and patterns

•  mind-maps summarise issues and patterns (not clickable as links)

•  issues and patterns are classified into sub-sections (e.g. management, process, etc.)

•  started 28 January 2013

76

About the patterns in the wiki •  the wiki has a limited set of patterns

– and a limited “depth” for each pattern •  patterns use other patterns

–  it can get rather complex •  we are focusing on system-level automation

rather than unit test automation – See Gerard Meszaros’ book “xUnit Test Patterns:

Refactoring test code” for unit test patterns •  a “work in progress”



77

Practical session •  explore the wiki with a partner •  take turns to help each other, using

– diagnostic questions –  issues – patterns

•  note patterns that may be useful to you – on the worksheet

•  then share with your group

78

Contents Background



The Wiki Conclusion




79

Feedback/discussion •  discussion of issues, patterns, wiki

– did you use the diagnostic questions? •  were they helpful?

– what issues & patterns did you look at? – what was most useful for you? – what else would be helpful? – any problems finding what you wanted? – how will you apply the patterns to your own

automation?

80

Using the wiki •  this is a new resource that will grow •  the more you add, the more valuable it will be

– please add your experiences of patterns, tips, problems, solutions

– email us with your ideas first •  will can put your text up •  or give you access to edit yourself

•  we welcome feedback about the wiki



81

Finally •  Thank you for coming today! •  SQE tutorial evaluation form

– vs the tutorial description on slide 2 •  Any questions now? •  Contact us afterwards:

–  [email protected] – [email protected]

•  All the best with your test automation!

Test Automation Patterns: Issues and Solutions

Technology