Page 1 Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final Prepared by Raj Kamal, [email protected]Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test Authors Raj Kamal ([email protected]) Vipul Mehta ([email protected]) Ranjit Gupta ([email protected]) Gunjan Jain ([email protected]) Microsoft Services - Global Delivery, India Reviewers Monty Pattan ([email protected]) Harish Reddy Kothapalli ([email protected])
66
Embed
Web Services Automation Guidance (vs 2010 Unit Test & Web Test)
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1
Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final
Test Approach – Identify your goal (Functional Test or Performance Test or Both) ................................... 4
Test Objective .................................................................................................................................................. 4
Choosing the appropriate Test Type (Unit Test or Web Test) ......................................................................... 5
Sample Lab Application: Web Service Automation ................................................................................... 6
Web Services Automation – Unit Test Walkthrough ................................................................................. 7
Creating a new Unit Test Project / Reusing Sample Template Unit Test Project ............................................ 7
Unit Test Automation Design & Development (WCF & ASMX based services) ............................................. 13
Web Services Automation – Web Test Walkthrough ............................................................................... 19
Creating a new Web Test Project / Reusing Template Web Test Project...................................................... 20
WebTest Automation Design & Development (WCF & ASMX based services) ............................................. 24
Web Service Performance Testing using Unit Test/Web Test + Load Test ............................................... 32
Web Services Helper Documentation for Unit Test & Web Test Add-ins ................................................. 38
Tools to speed up Web Services Testing .................................................................................................. 50
WCF Load Test ............................................................................................................................................... 50
Web Test Plug In for Silverlight + RIA based services .................................................................................... 54
Web Services Automation Benefits ......................................................................................................... 55
Web Services (WCF/ASMX) due to their worldwide implementation and popularity but
limited test tool support, often raises the question around testing them effectively.
Testers are craving to find effective ways to Test Web Services. Even VS 2010 doesn’t
have dedicated test types for verifying Web Services (like for UI, there is Coded UI and
for DB there is Database Unit Test) which results in low test coverage and non-
standardization of proven ways to test web services especially WCF 4.0 Web Services.
Web Services Testing in VS 2010 so far has been not been given a first-class citizen
treatment and this guidance will bridge that gap by making it much easier and effective.
In this agile development era, we are moving to a model where UI is ever changing and
consummation of web services is taking the centre stage and hence this layer attention
it truly deserves by shifting the onus from UI testing to Web Services based validation.
Traditionally teams have also used Fiddler and other 3rd
Party tools for the same purpose
(many times for manual testing alone) but as Web services have different needs, the
knowledge harvested doesn’t get reused to its potential in future engagements
Ideal Candidates
If your application is consuming APIs exposed by other upstream systems or 3rd Parties
If your application is exposing APIs to be consumed by the downstream applications
When your UI is dynamic and constantly changing and cost/maintenance of UI
automation is much more.
When your business logic and rules are primarily implemented using services (ASMX,
WCF & Workflow Services) and testing at this layer is much faster and efficient.
When you need to come up with a common automation framework for web services
that can be used to validate both functional and performance aspect of the web services
with minimum effort.
Test Approach – Identify your goal (Functional Test or Performance Test or Both)
When we understand and realize the importance of Web Services Automation, the next challenge is to select the right Test approach and Test Type before jumping into automation design and implementation.
Test Objective
It is important to answer the basic question “Why do you want to do Web Services Testing?”
and that will help you decide the right kind of Test you should choose to automate them.
Let’s begin with few possible answers
To validate the functional behaviour of your application / APIs
To validate the performance aspect of your application/ APIs
To first validate the functionality and then also test the performance aspect of it.
(a) + (b)
Page 5
Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final
This is an optional step if you don’t want to run your test from Microsoft Test Manager and map your automation to manual test cases. We however recommend this.
2. a
Have you created new Test project & Added it to Source Control (TFS)?
Every automation project must be source controlled.
Add project to Source Control by clicking the check box as shown and give appropriate name.
2.b Alternatively, Open existing Test project and using it as Template
Launch Visual Studio, Browse and open existing ‘WebServiceAutomation_SampleUnitTest’ template project and add it to Source control. You can change the properties as per your needs.
Page 9
Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final
2. Common Libraries References: Add references to our reusable helper library “WebServicesTesting.Automation.UnitTest.Helper.dll” for Unit Test Web Services Testing (provided as part of the release and documented below)
.
3. Service References: Add / update services reference to point to your web services and provide a logical name
4. Scripts: Your actual test are stored in these .cs files along with components like Test Initialize, Test Cleanup etc You can create a folder structure to organize your test methods by grouping them in multiple class files depending on the size of the project. It is advisable to group test belonging to same feature/functional module to be grouped together in a .CS file and given an appropriate name.
5. Solution Configurations & Compilation Symbols With our template solution you get additional solution configuraiton which will help you run your same test for both functional and load test seamlessly by controlling the sections of the code that you may want to execute for functional and load testing For example. When you do load test, you might not want to do certain validation which makes sense for functional testing alone and these configurations will control that by just selecting the appropriate solution configuratio as shown below.
a) Custom Solution configurations
Page 12
Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final
b) Configuration – Functional Testing As shown below it uses a compilation symbol #WEBSERVICES_FUNCTIONAL_TESTING that is used in our test methods to control the behavior.
c) Inside your test method, you will have code like this to control the behavior based on the solution configuration file that is selected
For more reference: http://www.codeproject.com/Articles/49448/Use-Visual-Studio-Conditional-Compilation-to-manag.aspx
Unit Test Automation Design & Development (WCF & ASMX based services)
Now that you have your test project in place and our sample reusable unit test cases for
automating common web services test scenario, in this section we will be walking you through
few sample reusable tests that you can customize it for your own needs or create new unit test
for your project scope. The approach shown below would remain same for WCF as well as ASMX
based services and we have shown examples of both.
Follow the below steps for Automation design guidance:
SL No.
Steps Description
1.
Global Settings
a) Set the test context which provides information about and
functionality for the current test run.
b) Test Initialize: To perform steps before running each test like instantiating objects, defining global variables including connection strings etc.
c) Test Cleanup: To run code after running each steps
2a. Add New Test Methods / Reuse Existing Template Test Methods
Sample Test scenario: Verifying Web service call response value against expected results
Below is sample data driven test method for web services testing that comes with our template project for Unit Test. You can add your new test methods by modeling the below example. Step 1.
Page 14
Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final
Create a new Test method / Modify our existing sample test method Step 2. Instantiate and invoke your web service method (in this example: ASMX based service) as shown in the code snippet below and get the results in a variable of appropriate data type In our sample example, the web service returns an integer datatype which we cast to String for comparison. Your services method can return any data type like int, double, decimal, bool, string etc and we can use the same helper method below to validate the response. Step 3. Get your expected results from data source of your choice as shown below.
Step 4.
Reuse our appropriate library / helper method for validating the web service response against expected results. In this example CompareString is a reusable method in our helper / library that can be called from your test methods to save effort related to validating the results and marking the test as pass/fail. Note: Depending on your test scenario, you need to choose different helper/library methods. The complete documentation about our reusable library method is given in sections below
2b
Add New Test Methods / Reuse Existing Template Test
Sample Test scenario: Verifying Web service call results in database operation like Insert
Below is sample test method when tester needs to validate if the web service call results into a Database Insert by validating the database for ensuring the transaction was committed. Instead of Insert, you might have different operations like update, delete etc.
Page 15
Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final
for which we have reusable helper methods in our library.
Here the web service method Enroll_Student (in this case WCF based Service) is inserting a record into the database and the expected result is the query against the database to validate the operation to ensure transaction was committed.
Reuse our appropriate library / helper method for validating the web service response against expected results. In this example Verify_DBInsert is a reusable method in our helper / library that can be called from your test methods to save effort related to validating the database insert and marking the test as pass/fail. Note: Depending on your test scenario, you need to choose different helper/library methods. The complete documentation about our reusable library method is given in sections below
2c
Add New Test Methods / Reuse Existing Template Test Methods (Handling Complex Object Types)
Use this approach as a generic way of comparing / validating web services response of various data types including user defined/abstract data types like Struct, Classes as well as primitive data types like Int, String, Dictionary, List etc. DTO (Data Transfer Objects) Having the results of a remote method
available in an object instance makes it easy to pass this data to a test module or to compare it with the desired outcome
It works by converting the service response (whether abstract or primitive data types) and expected results into standard generic objects which then are finally converted into XML for comparison with the help of our reusable library methods as described below in the example Sample Test scenario: Verifying Web service response which returns
Step 4. You can read your data source (in this case database) and
convert it into generic DTO object using reusable method in our library called DTO_PopulateDBResults which takes Dataset as input parameter. DTO Type should be same as the type of your complex object (in this case Student which is struct)
List<Student> ExpectedStudentDTOList = null; ExpectedStudentDTOList = helper.DTO_PopulateDBResults<Student>(dt); Step 5. In this step you call you Action code to get your web service response Actions.DTO_Action complexObj = new Actions.DTO_Action(); Object myObj = complexObj.ComplexObjectSample();
Step 6. Convert web services response into generic DTO using our
generic library method called DTO_PopulateObject which accepts Object and structure of the object defined above as parameters.
Step 7. Finally the generic DTO objects returned from expected
results i.e. Test data source and Web services response are converted into XML and compared using our reusable library method called DTO_XMLComparison and accordingly marks the test method as pass/fail
bool result = helper.DTO_XmlComparison(ExpectedStudentDTOList, ActualStudentDTOList, typeof(List<Student>));
Page 18
Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final
Assert.AreEqual(true, result, "Expected does not match actual"); Step 8. Expected and Actual XML files with the test method name are
created under your folder c:\TestResults for your future reference. Please create this folder if it doesn’t exist
3.
Make your scripts data-driven Irrespective of kind of test methods you have created above
Data-Driven Test Ensure every variable is parameterized by passing the test data to each test using data-driven capabilities in VS 2010. Right-click on the test method and select Properties, to view the properties of the test
To start making this a data-driven test, we have to first create a data source for the test to draw from. On the Data Connection String property, click the ellipse button to start the New Test Data Source Wizard There are three options for creating a data source: Database -- which allows you to connect to a database or Microsoft Excel spreadsheet CSV File -- which allows you to use a comma-delimited file XML File -- which allows you to use an XML file
Page 19
Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final
When you add a data source A Data Source attribute is added to the Test method. [DataSource("System.Data.SqlClient", "Data Source=(local);Initial Catalog=Sample_DB;Integrated Security=True", "Sample_App", DataAccessMethod.Sequential), TestMethod]
4. Map your automation test to your test cases in MTM
This is an optional step if you don’t want to run your test from Microsoft Test Manager and map your automation unit test to manual test cases. We however recommend this.
Repeat the above steps for all your test scenarios against your web services to be tested. After you write and debug your scripts its important to run them in a batch or as part of a suite. There are various way to do that and factors to keep in mind as we will see in next section.
Web Services Automation – Web Test Walkthrough
If “Web Test” was selected as the Test approach for Web service automation based on the analysis and
your test objective as covered in above section: Test Approach then proceed with the walk through.
Creating a new Web Test Project / Reusing Template Web Test Project
You can create a new project from scratch or using our existing sample template project but it’s
important to add your project to source control for version management and change tracking.
SL No.
Topics Description
1.
Have you created new Test project & Added it to Source Control (TFS)?
Every automation project must be source controlled.
Add project to Source Control by clicking the check box as shown
2. Alternatively, Open existing Test project and using it as Template
Launch Visual Studio, Browse and open existing “ WebServiceAutomation_SampleWebTest” template project and add it to Source control. You can change the properties as per your needs.
Page 21
Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final
Please find below a short description on the folder structure of the template project which will be used for case study:
1. Common Libraries References:
a) Add references to our reusable helper library “WebServicesTesting.Automation.WebTest.Helper.dll” for WebTest based Web Services Testing (provided as part of the release and documented below)
Page 22
Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final
2. Service References: Add / update services reference to point to your web services and provide a logical name
3. Scripts: Your actual test are stored in these .webtest files.
You can create a folder structure to organize your test methods by grouping them in multiple class files depending on the size of the project. It is advisable to group test belonging to same feature/functional module to be grouped together in a .CS file and given an appropriate name.
For this sample template project we have many reusable webtest that can be reused / customized for your needs.
Page 24
Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final
Step 6. Add our reusable Validation and Extraction Rule called DatasetValidation that reads the expected data from database using SQL query and validate it against the response from web services by using XPathExpression
Note: Depending on your test scenario, you may need to choose different Extraction and Validation Rule complete documentation about our reusable library method is given in sections below
1b. Add New Web Test / Reuse Existing Template Web
Sample Test scenario: Verifying Web service method parameters and response values against expected test data.
Page 27
Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final
Test (Method Prototype Verification for WCF/ ASMX Service)
Repeat Step 1 to 5 from 1 a) above as explained
Step 6. Add our reusable Extraction Rule called ParameterExtraction and provide XPathExpression for example Enroll_Student and it will retrieve all the parameters name and their respective values and store it inside ContextParam
Step 7. Step 6. Add our reusable Validation Rule called ParamValidation and provide ParamValues that contain expected test data as shown below along with contextParam name as defined in Step 6.
Page 28
Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final
Step 8 & 9. As shown in this web test you can also add our reusable ServiceResponseExtraction and ServiceResponseValidation rules from our helper class for validating the response values against expected test data. Step 8. After you run this web test, the result will show Pass / Fail with context tab showing the discrepancies if any.
Note: Depending on your test scenario, you may need to choose different helper/library methods. The complete documentation about our reusable library method is given in sections below
2.
Make your scripts data-driven
Data-Driven Test Ensure every variable is parameterized by passing the test data to each test using data-driven capabilities in VS 2010.
Page 29
Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final
To start making this a data-driven test, we have to first create a data source for the test to draw from. On the Data Connection String property, click the ellipse button to start the New Test Data Source Wizard There are three options for creating a data source: Database -- which allows you to connect to a database or Microsoft Excel spreadsheet CSV File -- which allows you to use a comma-delimited file XML File -- which allows you to use an XML file
Repeat the above steps for all your test scenarios against your web services to be tested. After you write and debug your scripts its important to run them in a batch or as part of a suite. There are various way to do that and factors to keep in mind as we will see in next section.
Ensure you have configured test settings to capture logs, code coverage, test impact etc. and you also need to keep in mind that your script should be able to successfully run in multiple environment (for ex. Test to UAT)
Page 30
Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final
After automation design and debugging is completed, you can now run and deploy your automation scripts using the below steps:
SL No.
Areas Description
1.
Configuring Test Settings
Test Settings
a) Data and Diagnostic: Enable appropriate diagnostic are selected as shown below (Code Coverage, Event Log, Test Impact etc.)
b) Define other settings like Roles, Test Timeouts, Setup and Clean scripts
2.
Command Line Execution (MSTest.Exe)
Grouping & Classifying Test Methods using various parameters It lets you run groups of tests based on their assigned categories without the requirement to maintain test lists.
Go to Test View and you can filter and view your tests by setting these parameters
Page 31
Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final
MSTest.EXE Options: MSTest.exe is the command-line command that is used to run tests.
This command has several options that you can use to customize your test run. You can specify these options in any order on the MSTest.exe command line.
Refer to http://msdn.microsoft.com/en-us/library/ms182489.aspx
3. TCM.exe or Test Execution from Microsoft Test Manager (Optional though recommended)
a) If you have mapped you automation unit test with test cases in Microsoft Test Manager then you can use TCM.exe as shown below to run your web service test and it will also update your test cases results in MTM without any custom code. You don’t need to use MSTest.exe in that case
Example: tcm run /create /title:title /planid: id /collection:CollectionURL /teamproject:project (suiteid:id /configid:configid | /querytext: query) [/settingsname:name] [/owner:owner] [/builddir:directory] [/testenvironment:name] [/login:username,[password]] [/include]
Refer to : http://msdn.microsoft.com/en-us/library/dd465192.aspx &
http://msdn.microsoft.com/en-us/library/ff942469.aspx b)You can also Run Automated Tests from a Test Plan Using Microsoft Test Manager. To run the latest version of your automated test, you must make sure that the correct build is selected for your test plan. You must also use a physical or virtual environment to run your automated tests. This is out of scope of this guidance however we recommend this the most. Refer to: http://msdn.microsoft.com/en-us/library/dd553270.aspx
Please follow Adding custom counters set using Load Test Editor for detailed steps. Step 1. Open you load test. Right click you counter set and add “Custom Counter Sets” Step 2. Right click and now click “Add Counters” Select Performance Category to include counters related to Web Services as shown below
Step 3. Repeat it for all the categories as shown below.
Tip: You can enable performance counters for a WCF service through
the app.config configuration file of the WCF service as follows: <configuration>
Category Title Description Unit Method Signature & Example invoke call
Web Custom Extension (Extraction & Validation Rules)
Lookup Verification
Verifies the Scalar value or multiple scalar values of even different data types returned by services against expected results
This method will let you compare the scalar values of different types like <INT>, <STRING> etc. returned by the service method call converting them to <string>against the expected values
For scalar value Extraction Rule ServiceResponseExtraction Context Parameters 1.XpathExpression 2.ContextParam Validation Rule ServiceResponseValidation Context Parameters 1.Expected 2.ContextParam For multiple values Validation Rule DataSetValidation Context Parameters 1.ConnString 2.Query 3.XPathExpression 4.ContextParam The validation rule extracts the dictionary returned by web service from xml and compares with the expected dataset based on the query
Lookup Verification
Verifies the values of <DICTIONARY> data type returned by services against the expected results
This method will let you compare the <Dictionary> <key, value> values returned by service method call against the expected values <key, value>.
public void CompareDictionaryWithExpectedArray(Dictionary<int, string> ActualDic, string[,] ArrgetAgeExpected) Example : obj.CompareDictionaryWithExpectedArray(col_Symbols, ArrgetAgeExpected);
Validation Rule DataSetValidation Context Parameters 1.ConnString 2.Query 3.XPathExpression 4.ContextParam The validation rule extracts the dictionary returned by webservice from xml and compares with the expected dataset based on the query
Page 40
Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final
Verifies the values of <LIST> data type returned by services against the expected results
This method will let you compare the values of data type <LIST> returned by service method call against the expected values
public string[] CompareList(string query) Example: string[] ArrgetAgeExpected = obj.CompareList(query);
Validation Rule DataSetValidation Context Parameters 1.ConnString 2.Query 3.XPathExpression 4.ContextParam The validation rule extracts the List returned by webservice from xml and compares with the expected dataset based on the query
Lookup Verification
Verifies the result set returned by services against expected results
This method will let you compare the result set returned by service method call against the expected result set
Method Prototype public void Verify_Dataset(string ConnString, string Query, DataSet dst) Method Invocation UnitTest help = new UnitTest(); help.Verify_Dataset(conn, query, set);
Validation Rule DataSetValidation Context Parameters 1.ConnString 2.Query 3.XPathExpression 4.ContextParam The validation rule extracts the dataset returned by webservice from xml and compares with the expected dataset based on the query
Lookup Verification
Verifies the row count of the values returned by Web Services against expected result
This method will let you validate the row count of the values/list/dataset returned by service method call against the expected row count
Method Prototype public void Verify_RowCountDataSet(int expected, int actual) Method Invocation UnitTest help = new UnitTest(); help.Verify_RowCountDataSet(expected, actual);
Extraction Rule RowCountExtraction Context Parameters 1.XpathExpression 2.ContextParam This method extracts the row count and stores the value to Context Param specified by the user. Validation Rule RowValidation Context Parameters 1.ConnString
Page 41
Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final
2.Query 3.ContextParam 4.ContextParam This method validates the value set to the context param and the one returned by the specified query
Metadata Verification
Verifies the called web service method has schema as expected
This method validates the XML file to make sure it fits to your application's needs to prevent any exception in your code. To check whether the XML document conforms to an XSD Schema, the document must be validated against that XSD Schema
Method Prototype public bool Verify_Schema(string targetNameSpace, string xsdPath,string requestXml) Event Handler to raise error private static void ValidationEventHandler(object sender, System.Xml.Schema.ValidationEventArgs arg) Method Invocation UnitTest help = new UnitTest(); bool validation = help.Verify_Schema(targetNameSpace ,XsdPath,requestXml);
Verifies that web service methods exist and their parameters definition (like input parameters, return type) against expected result
This method will let you smoke test / BVT if the expected list of Web Service methods exists and their definition like input parameters and return type matches.
Method Prototype public void Verify_methodPrototype(object myObject, string Methodname, string paramlist, string returnTypelist) Method Invocation UnitTest help = new UnitTest(); help.Verify_methodPrototype(obj, methodlist, paramlist, returnTypelist);
Extraction Rule ParameterExtraction Context Parameters 1.XpathExpression 2.ContextParam This method extracts the parameter passed to the request and set to the contextparam Validation Rule ParamValidation Context Parameters 1.ParamValues 2.ContextParam
Page 42
Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final
This method validates the value set to the context param and the Values passed to the ParamValues The above extraction & validation rule is used to validate the parameter passed The below extraction & validation rule is used to validate value returned by the webservice Extraction Rule ServiceResponseExtraction Context Parameters 1.XpathExpression 2.ContextParam Validation Rule ServiceResponseValidation Context Parameters 1.Expected 2.ContextParam
Transactional Verification
Verifies that called web service method creates the output file as expected
This method verifies the output file (e.g. CSV, XML, TXT) has got created by the web service method call
Method Prototype public void Verify_FileCreation(string FilePath) Method Invocation UnitTest help = new UnitTest(); help.Verify_FileCreation("D:\\employees.xml");
Verifies that web service INSERTS/CREATES a new record into the database
This method verifies the web service call results into new record insertion in a given database
Method Prototype public void Verify_DbInsert(string ConnString, string Query, string expected) Method Invocation UnitTest help = new UnitTest(); help.Verify_DbInsert(conn, query, expected);
Extraction Rule ParameterExtraction Context Parameters 1.XpathExpression 2.ContextParam This method extracts the parameter passed to the request and set to the contextparam
Page 43
Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final
Validation Rule DbInsertValidation Context Parameters 1.ConnString 2.ContextParam 3.Query 4.ContextParam This method validates the contextparam and the values returned by the query
Transactional Verification
Verifies that web service UPDATES existing records into the database
This method verifies the web service call results into update of existing record(s) in the given database
Method Prototype public void Verify_DBUpdate(string ConnString, string Query, string expected) Method Invocation UnitTest help = new UnitTest(); help.Verify_DBUpdate(conn, query, expected);
Extraction Rule ParameterExtraction Context Parameters 1.XpathExpression 2.ContextParam This method extracts the parameter passed to the request and set to the contextparam Validation Rule UpdateDbValidation Context Parameters 1.ConnString 2.ContextParam 3.Query 4.ContextParam This method validates the contextparam and the values returned by the query
Transactional Verification
Verifies that web service DELETES records into a database
This method verifies the web service call results into deletion of existing record(s) from a given database
Method Prototype public void Verify_DBDelete(string ConnString, string Query) Method Invocation UnitTest help = new UnitTest(); help.Verify_DBDelete(conn, query);
Validation Rule DeleteDbValidation Context Parameters 1.ConnString 2.Query This method verifies the rows returned by query is 0
Page 44
Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final
Workflow service Verify that web service executed the workflow and end result is as expection
_________For this one we just need a test method in our test file instead of a separate helper method____
Sample code to verify the result returned by the workflow IDictionary<string, object> _d = new Dictionary<string, object>() { { "userArgument", arg } }; _d = WorkflowInvoker.Invoke(new Workflow1(), _d); Int32 outPut = (Int32)_d["OutputValue"]; Assert.AreEqual(expected, outPut); output can be validated with expected for its correctness
WorkFlow cannot be invoked. We can invoke the web service to verify the result returned by service method against the expected result. Extraction Rule ServiceResponseExtraction Context Parameters 1.XpathExpression 2.ContextParam Validation Rule ServiceResponseValidation Context Parameters 1.Expected 2.ContextParam
Authentication Verifies that user credentials & certificate are validated against the server to authenticate the user trying to access the web service
This method verifies the user credentials and certificate to ensure that he is authenticated to get the results from the web service
var ServiceendPoint = new EndpointAddress(new Uri("http://localhost:91/Service1.svc"), EndpointIdentity.CreateDnsIdentity("MyWebSite")); var binding = new WSHttpBinding(); binding.Security.Mode = SecurityMode.Message; binding.Security.Message.ClientCredentialType = MessageCredentialType.UserName; var result = new Auth_Service.Service1Client(binding, ServiceendPoint); try {
Page 45
Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final
Auth_Service.Service1Client result = new Auth_Service.Service1Client(); result.ClientCredentials.ServiceCertificate.Authentication.CertificateValidationMode = X509CertificateValidationMode.None; result.ClientCredentials.UserName.UserName = auth_username; result.ClientCredentials.UserName.Password = auth_passwd; string time = result.GetServertime(); TestContext.WriteLine("Bindings used by this contract is " + binding_used);
Caching Verifies that data returned by called web service is returned from cache
_________For this one we just need a test method in our test file instead of a separate helper method____ Caching is frequently used as an optimization in distributed systems. It can be used to avoid re-doing computations or complex database access when the results remain valid for an extended period of time.
string url = "http://localhost/Cache/Service1.svc/xml/12"; Uri uri = new Uri(url); WebRequest request = (HttpWebRequest)WebRequest.Create(uri); request.Method = "GET"; HttpWebResponse response = (HttpWebResponse)request.GetResponse(); Stream receiveStream = response.GetResponseStream(); StreamReader readStream = new StreamReader(receive
Extraction Rule CacheExtraction Context Parameters 1.XpathExpression 2.ContextParam 3.Delay To test caching two request should be made before caching expire and one after caching expire. Parameter Delay has to be passed for the second request And plugin DelayPlugin should be associated with the second service CacheOutcomePlugin ContextParameters 1.ContextParam1 2.ContextParam2 3.ContextParam3
Page 47
Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final
In this case, subsequent requests for the same information can be served with the cached version rather than repeat the processing with the associated overheads.
Stream, Encoding.UTF8); string firstResponse = readStream.ReadLine(); Above code snippet shows to retrieve server response. To test caching two request should be made befor cache expires and should be validated.
Caching Verifies that data returned by called web service is returned from database when cache is expired
_________For this one we just need a test method in our test file instead of a separate helper method____
string url = "http://localhost/Cache/Service1.svc/xml/12"; Uri uri = new Uri(url); WebRequest request = (HttpWebRequest)WebRequest.Create(uri); request.Method = "GET"; HttpWebResponse response = (HttpWebResponse)request.GetResponse(); Stream receiveStream = response.GetResponseStream(); StreamReader readStream = new StreamReader(receiveStream, Encoding.UTF8); string firstResponse = readStream.ReadLine(); Above code snippet shows to retrieve server response. To test caching two request should be
Page 48
Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final
made before caching expire and one after caching expire and should be validated.
Non-XML Data Verifies that called web service method sends/returns non-xml data (like blob, image) as expected
This method will verify that web service method is storing or retrieving the non XML data like blob from file system or database
Refer our reusable [TestMethod] public void VerifyNonXMLData()
Asynchronous messaging
Verifies that called web service method if implemented as asynchronous then returns the data as expected at a later point in time
This method verifies the asynchronous behavior of web service method as expected and implemented.
Refer our reusable [TestMethod] public void VerifyAsync_Response()
Exception and Error handling
Verify that service returns appropriate error message when there is a fault exception like underlying database connectivity issue to ensure exception and error handling
This method will verify that web service method that is called handles the exceptions like timeout, connectivity/networking issue as expected
<serviceDebug includeExceptionDetailInFaults="True" /> Set the IncludeExceptionDetailInFaults property to true to instruct to return managed exception information to the client in SOAP faults to ease debugging try { WCFServiceRef.Service1Client obj = new WCFServiceRef.Service1Client(); obj.Display();
Extraction Rule ExceptionExtraction ContextParameter 1.contextParam Validation Rule ExceptionValidation ContextParameter 1.contextParam 2.ErrorMsg ExceptionPlugin ContextParameter 1.contextParam Plugin is used to pass/fail the test case contextParam name provided in extraction/validation/plugin
Page 49
Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final
Merge Verifies the result obtained from multiple web services calls by merging them matches against consolidated expected output
This method verifies when similar result is obtained from multiple web services and then need to be validated against single expected result set Think of a scenario where you are getting similar result from various source - each a different web service call. The tester wants to validate the consolidated output from these services by comparing it against expected values
Extraction Rule ServiceResponseExtraction Context Parameters 1.XpathExpression 2.ContextParam The extraction rule is used with both the request to extract the response from the server Associate the validation rule with the 2
DTO (Data Transfer Objects) based validation is a generic way to validate web services response of any data type against expected result by converting them into generics
It works by converting the service response (whether abstract or primitive data types) and expected results into standard generic objects which then are finally converted into XML for comparison with the help of our reusable library methods as
public List<T> DTO_PopulateObject<T>(object DTOObject, object myObj) public List<T> DTO_PopulateDBResults<T>(DataTable dt) where T : class public bool DTO_XmlComparison(Object ExpectedDTOList, Object ActualDTOList, Type ObjectType)
Page 50
Web Services Test Automation Guidance using VS 2010 - Unit Test & Web Test, Design Document, Final