From Unit Testing to Load Testing : Visual Studio 2010-Your Complete Testing tool 19 Aug 2011 SANDEEP JOSHI AVP Technology, Barclays Capital @sandeepmvp http://blogs.msdn.com/sajo shi
Dec 24, 2015
From Unit Testing to Load Testing : Visual Studio 2010-Your Complete Testing tool
19 Aug 2011
SANDEEP JOSHIAVP Technology, Barclays Capital
@sandeepmvp http://blogs.msdn.com/sajoshi
Agenda Developer’s Role in Quality Options for automated tests Automated Functional Testing
Coded UI Test Demo
Load Testing− Web Performance Tests− Load Tests− Load Test Rigs− Performance Explorer
Developer’s Role in Quality
− Find Problems before you make them− Code Analysis− Code Metrics− Performance Tests
− Don’t let bugs out of your sight− Unit Testing and Code Coverage− Test Impact Analysis− Coded UI Tests
− Or into your builds− Gated Check-In
Dose Of Reality
− You know you should work that way.− You probably don’t.− This talk is for you.− You have problems.− How do you figure out what’s going
wrong?
It’s not that difficult as you think.
Options for automated tests Several kinds of automated tests
DatabaseBusiness Logic
User Interface
DatabaseUnit Tests
T T T T T
Unit Tests
T T
T T T
T T
Web Performance
Tests
T
T
T
T
Coded UI Tests
T
T
Automate Functional Testing
70%of all functional testing is still done manually.
Automate Testing - Coded UI Tests
− Automate functional tests.
− Record actions or import action recording from test.
− Easily repeatable.
Coded UI Test
demo
Load Testing:
Disclaimer: I’m going to use the term “load testing” in most general sense.
Load Testing – Goals
− Identify the capabilities of your application
− Possible goals− Explore− Verify − Find the limits− Crush
The Tools
Visual Studio
Ultimate
• Web Performance Tests• Load Tests• Load Test Rigs
Visual Studio Premiu
m
• Profiling Tools• Performance Wizard
Disaster Strikes.
− The application is deployed to production.
− Users are losing their minds.− Intermittent errors on the site.− Site is slow.
− Your boss is seriously angry.− “Fix it!”
The Problem.
− “No Repro”− “Works on my box.”− You don’t know what the errors are.− The site seems to work fine.
− Wild guess: it dies under load.− You need to create some load.
The Plan.
− Script user activity with Web Performance Tests
− Simulate a mix of users using Load Tests
− Simulate lots of users with a Load Test Rig
− Reproduce the errors...hopefully− Find code problems with the
Performance Wizard− Fix the code
What is a Web Performance Test?− Test type in Visual Studio 2010
Ultimate− Simulate a user using an ASP.NET
application− HTTP traffic
− Building block of Load Tests
Tip: Web Tests & User Stories
− Web Tests should simulate a User Story
− Example:− As an administrator, I need to search for a
person by username so that I can deactivate that person’s account.
− As a customer, I want to order a pizza and pay with a credit card.
− Helps to organize your suite
Data-driven Web Tests
− Run web test once per row in data source
− Simulate multiple users doing *similar* actions
− Data source types− OLE DB, CSV, XML
− Parameterization− Better tests, better coverage
What are Load Tests?
− Mix of Web Performance and Unit Tests
− Simulates a mix of users doing a mix of things
− Simulate − Network speeds− Different browsers− Varying user activity loads
Unit Test in a Load Test?
− Helpful for testing WCF− Cumbersome to simulate WCF via
Web Test− (You’d have to re-invent the wheel.)
− Unit tests can re-use the WCF libraries− (Done and done.)
− Important for Silverlight and SOA− http://wcfloadtest.codeplex.com/
− http://sqlloadtest.codeplex.com/
Demo
− Creating and Running Load Test with VS2010
Two ways to run load tests.
− Generate load from Visual Studio 2010− Uses only 1 core/processor− Limited to 250 simulated users
− Generate load from a Test Rig− Far better test− Simulate lots more users
Load test Rigs & Application
Web Performance
Tests
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
Test Agent
Test Agent
Test Agent
Test Agent
Test Controlle
r
ControllerOrchestratesLicensingPerfMon Counters
Agent (1..n)Simulates users
Application Under Test
Application Under
Test
Generates Load
Coordinates &
Gathers Informatio
n
Controller
Agent 1 Web Server
DatabaseAgent 2
Agent n
Load Test Agent Licensing
− Microsoft Visual Studio Team System 2010 Test Load Virtual User Pack
− Simulate up to 1000 virtual users− Used to cost $$$
Visual Studio 2010 Load Test Feature Pack− Used to cost ~$3500.00 per 1000
simulated users− Free starting March 8, 2011− Visual Studio 2010 Ultimate with
MSDN− Simulate unlimited amount of load
− (Limited only by your hardware.)− http://www.microsoft.com/
visualstudio/en-us/products/2010-editions/load-test-virtual-user-pack
Profiling Tools
− Load Tests help you find issues in a system− “Surface” issues
− Profiling helps you find issues in the code
− Unit Tests, Web Tests, Load Tests− Repeatable− Good for targeted tuning
− System under “natural” load− Not repeatable− Looking for rough patterns
Profiling Methods• Gathers information at intervals (clock
cycles)• No code modifications• Use this for first pass explorations
Sampling
• Modifies your code• Gathers detailed timing and count
information
Instrumentation
• Multi-threaded code profilingConcurrency
• Object creation• Garbage collection.NET Memory
• Interaction between your application and SQL Server via ADO.NETTier Interaction
Additional Resources
Visit Microsoft’s website:www.microsoft.com/visualstudio/test
Channel 9:http://channel9.msdn.com/learn/courses/vs2010
Team Blogs:http://blogs.msdn.com/amit_chatterjeehttp://blogs.msdn.com/vstsqualitytoolshttp://blogs.msdn.com/edglashttp://blogs.msdn.com/lab_management
Feedback
9
© 2009 Microsoft Corporation. All rights reserved. Microsoft, Visual Studio, the Visual Studio logo, and [list other trademarks referenced] are trademarks of the Microsoft group of companies.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the
part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.