Test Case Passed! Who Cares!!! - Pradeep Soundararajan
Post on 07-Apr-2018
219 Views
Preview:
Transcript
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
1/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
Hello!
I am Pradeep Soundararajan from Bangalore, India
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
2/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
I test, manage, coach, consult, speak, write on software testing.
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
3/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
Here is a story of how I struggled to find bugs a few years
ago, which I thought might be of help to you.
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
4/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
No, this is not a story without a name, so here is the name
The snake in the monkeys shadow
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
5/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
oOPS! I think that wasnt a title that makes sense to this
context. OK, Let me rename it
Test case Passed! Who cares?
The title will appear above the above text in 45 seconds !
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
6/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
To start the story, here is an expected result of a test case
FR_1223 that I executed!
The application should handle the input andshould not hang or crash the application.
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
7/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
I executed that test and passed the test because the
application did handle the input and did not crash/hang the
application.
Are you curious to know what happened once I
passed the test?
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
8/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
The customer complained that although the input was
handled, the application took 15 seconds to perform the
operation.
but hey, the Spec didnt say that?
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
9/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
Well the test passed but why did I not catch the bug
reported by the customer?
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
10/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
I then added another point to the expectedresult:
Expected Result of test case FR_1223:
The application should not take more than 5 seconds tohandle it.
The application should handle the input and should not hangor crash the application.
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
11/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
I executed the test again in the next release and the test
passed!
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
12/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
Yet again, the customer came back logging another issue:
While the application now takes 5 seconds to handle, it fails
to process any input once I close and reopen the applicationafter an abrupt closure during those 5 seconds.
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
13/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
I then updated the Expected Result of test case FR_1223 to:
The application should handle any abrupt closure orthere should be a proper warning or error messageduring that operation.
The application should handle the input and should not hangor crash the application.
The application should not take more than 5 seconds tohandle it.
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
14/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
And in the next build I executed the test and it passed!
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
15/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
Our customer came back logging another issue: While the
application now handles abrupt closure by warning the user
of loss of data, the database elements are corrupted whenthe operation is iterated 5 times.
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
16/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
I then updated the Expected Result of test case FR_1223 to:
When such an abrupt closure is carried out multiple timesthe database should not be corrupted.
The application should handle the input and should not hang or crashthe application.
The application should not take more than 5 seconds to handle it.
The application should handle any abrupt closure or there should be aproper warning or error message during that operation.
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
17/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
Our customer was quite happy that he could not find
anymore issues with respect to that feature and most of his
expectations were met.
Wow! The story appeared to end.
I was simply wrong!
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
18/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
Our customer then passed the release to their customer. Our
customer came back saying their customer isnt happy
because the Submit button that performs an action ofprocessing the input is too small to be noticed for new users.
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
19/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
I then updated the expected result to:
Expected Result of test case FR_1223:
All radio buttons associated with this operation should be ofsize X by X (as communicated by the customer in e-maildated October 22)
The application should handle the input and should not hang orcrash the application.
The application should not take more than 5 seconds to handle it.
The application should handle any abrupt closure or there should be
a proper warning or error message during that operation.
When such an abrupt closure is carried out multiple times thedatabase should not be corrupted.
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
20/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
And in the next build I executed the test and it passed!
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
21/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
Yet again, our customers customer, reported an issue: It
appears that when this operation is being processed in IE,
the browser shows Not Responding and then recovers.
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
22/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
I then updated the expected result to:
Expected Result of test case FR_1223:
The application should process the operation in a same way acrossmultiple browsers ( IE, Firefox, Safari, Web Monkey, Netscape )
The application should handle the input and should not hang or crash theapplication.
The application should not take more than 5 seconds to handle it.
The application should handle any abrupt closure or there should be a properwarning or error message during that operation.
When such an abrupt closure is carried out multiple times the database should
not be corrupted.
All radio buttons associated with this operation should be of size X by X ( ascommunicated by the customer in e-mail dated October 22 )
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
23/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
The story went on for a while and today when I look back, I
realize I learned a few important lessons in testing.
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
24/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
I am wondering if I should share the lessons with everybody
instead of people whom I choose to share?
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
25/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
OK, so enter the password that I might have sent youto read the lessons, else you dont get to read them
and I am sorry about it.
Password: Check
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
26/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
Wait a minute! I changed my mind.
You can read them without entering the password
and here it goes
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
27/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
Lesson Number Minus Three
The specification could have been met long back but it is not the
specification that approves the product nor the ones who wrote
the specification but it is *humans*- who arent completely
bothered if specification is met but bother if it meets their
expectations without bothering if their expectation was
documented in the specification or not.
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
28/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
If you think your customer learns new things every day as you do,
then you must be aware that their expectations keep changing as
they learn, just like how it is happening for you.
Do you expect the same salary once you have learned that your
contribution to a project that you work has grown humongous?
If you expect the same you deserve the same
Lesson Number Minus F
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
29/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
If all expectations were to go in to a column of test case whose
heading is Expected Result then each element of the column
might be of several thousand words and hundred points as compared
to what we most commonly get to see even if we were to writethose test cases.
It might take about an hour just to read such a test, isnt that funny?
Lesson Number Minus Five
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
30/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
Lesson Number PLUS six
(Were you expecting a MINUS six? )
When I practiced using heuristics* and oracles** (not the database)while exploratory testing a product I found that I loaded my brain withthe list of oracles and heuristics and then shot every test with as many
expectations I could think of and I was able to find not just one bug withevery test I did but many.
Heuristic* A fallible method of solving a problem
Oracle** - A principle of mechanism by which we (humans and not the process)
identifies a problem
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
31/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
Lesson number minus seven-teen
I recommend other testers to not use the Heuristics and Oraclesapproach to their testing because I am a little selfish about being oneof the few testers who finds many important problems, quickly.
I also dont recommend you to go through Rapid Software Testingmaterial that helps you in learning a lot more about Oracles andHeuristics which can be found at www.satisfice.com/rst.pdf
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
32/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
Lesson number
eight If I had all lessons spoon fed tome, I would have become thelaziest tester in the world and I
think I did not become thatbecause I realized the one of thesolid ways to do a good testing, isby practicing doing good testing.
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
33/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
Lesson number Aight!
When I execute tests that have words like handle theinput looks reasonable correct results are
displayed
My opinion about those words might be different fromthose of my customers and even other testers.
Lesson contribution from Ben Simo www.questioningsoftware.com
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
34/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
Lesson number 8 ( not the word eight )
A different lesson dont worry about that!
I think we learned so many things with the story and Iguess we should have also learned that it is humanswho solve problems and not the process they make,which many think takes care of testing.
A process is a heuristic and can never be a bestpractice!
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
35/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
The first time I heard Heuristic and Oracles
I had the same feeling as you might be having now Greek andLatin
but I asked myself a question, If I can do a lot better testing by
learning that Greek and Latin, why not I learn it?
and here I am writing stories and articles about them after exploringthe basic power of those two words.
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
36/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
Thank you for your time!
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
37/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
This presentation is inspired by Michael Boltons Emotions and
Oracles Tutorial and the creative (in my opinion) style of presenting
was inspired by Ben Simos FAILURE presentation (that is not
public yet) and from the titles of the movie Monty Python and theHoly Grail.
I admit that I didnt do anything more creativethan those!
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
38/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
Some of you might have thought, after going through this
presentation (so far) that I am trying to say, I no longer struggle to
find bugs because I exploit the power of Heuristics and Oracles.
I am glad to say to those who thought that way,
I think you are wrong, I still do struggle but I might not struggle on
the same problems that I struggled last year or maybe yesterday
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
39/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
For more such stuff that might interest / bore you like this keep
visiting my blog http://testertested.blogspot.com
Lessons are not for free they are mixed with some
marketing, so bare with it
8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan
40/40
Copyrights Pradeep Soundararajan http://testertested.blogspot.com pradeep.srajan@gmail.com
The END!
http://testertested.blogspot.com pradeep.srajan@gmail.com
"Pradeep's first language is not English--his first language appears to betesting." -- Michael Bolton
top related