Top Banner
Randy Pagels Sr. Developer Technology Specialist Microsoft Corporation www.teamsystemcafe.net Virtualizing your testing with Visual Studio Lab Management
56

Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Mar 24, 2018

Download

Documents

vantuyen
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Randy PagelsSr. Developer Technology SpecialistMicrosoft Corporation

www.teamsystemcafe.net

Virtualizing your testing with Visual Studio Lab Management

Page 2: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Modern app development Continuous value delivery and rapid response to change

Page 3: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine
Page 4: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

User

Interface

Services

Business Processes

Business Rules and Logic

Service integrations

Data access

Identity

Data

Automated testing

Manual testing

Automated testing

Page 5: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Lab Management Overview

Page 6: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine
Page 7: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine
Page 8: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Lab Management Basics

Environments Create and manage virtual or physical dev/test

environments

Snapshots Take snapshots or revert to existing snapshots

Works on any virtual machine

Deploy current or any build to environment

Environment Viewer Interact with machines through the environment viewer

Define test settings for environments

Page 9: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Leveraging Virtualization

Page 10: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Lab management services

Environments

On-premise | Cloud

Page 11: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Benefits of Virtualization

TestingVirtual test environments can be used for

Testing CI builds using GUI automation

DebuggingDebug hard to reproduce defects

Maintaining snapshots of older builds

DemosDemos for customers

Test Production hot-fixes

Load Testing

Page 12: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Leveraging virtualization for ALM

Page 13: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Testing Architecture

VM Template Library SCVMM Environment Standard Environment

VM Template 2VM Template 1

Test Agent Test Agent Test AgentTest AgentVMMAgent

Web Server

VMDatabase Server

VMVM Host Web Server Database Server

Test AgentTest Agent

Team Foundation Server

System Center Virtual Machine Manager (SCVMM)

Team Build

Test Controller

Page 14: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Build-Deploy-Test

SCVMM Environment

Test AgentTest AgentVMMAgent

Web Server

VMDatabase Server

VMVM Host

Team Foundation Server

Test Controller

Team Build

(1) Get Source Code (2) Compile projects

(7) Send test

results

(6) Execute

automated tests

(3) Copy build to running

environment

(4) Run deployment scripts for

each machine

(5) Create environment snapshot

(8) Publish results to

Team Foundation Server

Page 15: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Environment DifferencesSCVMM Environment Standard Environment

Build-Deploy-Test

Snapshot / Restore

Share Bug Snapshots

Network Fencing

3rd Party Virtualization

Physical Machines

Page 16: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Test Controller

Run automated tests remotely

Distribute automated tests to multiple

environments

Run tests and collect test data remotely

using Microsoft Test Manager

Run tests in a lab environment

Deploy applications using a build-deploy-

test workflow

Page 17: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Test AgentsManage testing, workflow, and network

isolation

Scale out load generation

Distributed data collection

Distributed test execution

Automatically installs test environments

Can be manually installed/maintained

using the VMPrepTool

Page 18: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Sharing and Libraries

IT Administrator Tester

Import VMs

Create Environments

Test in Environments

Store Environments

Manage virtual machine infrastructure

Create, deploy, and manage virtual machines

Monitor and manage hosts/library servers

Manage global configuration settings

Save Environments in Library

Create Active Environments

VM Template Library

Test Agent

SysPrepped “golden images’

Win Server 2008

Win Server 2012

Win XP SP 3

Win Vista SP 2

Win 7 SP 1

Win 8

OS Support

4 machine environment

1) Domain Controller

2) Client VM

3) Web Server VM

4) Database Server VM

Self Provisioned VM Environment

Page 19: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Lab Management

Page 20: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Save costs by

virtualizing your

labs

Page 21: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Licensing

Lab Management is per CPU license for the hosts you want to manage

For client you need either Ultimate/Test Professional

SCVMM license for dev/test is included.

Guest OS license is part of MSDN subscriptions

VMWare Support

Support through use of Standard Environments

Page 22: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Resources

Page 23: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Visit Team System Café Today!The door is open 24/7 at the café! Stop by for a cup of joe and delicious tidbits on AppDev topics!

http://www.teamsystemcafe.net

Page 26: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Using a Lab Environment for Your Application Lifecycle

Getting Started with Lab Management

Guidance for Creating and Managing SCVMM Environments

Cloning Virtual Machines through Network Isolation

Creating Lab Environments

Managing Lab Environments and Virtual Machines

Testing for Continuous Delivery with Visual Studio 2012

Setting Up Automated Build-Deploy-Test Workflows

Lab Management Resources

Page 27: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Best Practice Topology GuideTopology 1 - Multiple AT, Load Balancer and Test network with firewall settings, controlling the test traffic in and out of Corp network

Network load balancer

Firewall rules

Topology 2 - Multiple ATs and DTs without load balancers and Test network with SAN based library and host

SAN support

Topology 3 - TMG, Windows NLB and with Test apps having DB tier outside of virtual environment

Setting up rules in Threat Management Gateway

Deployment and testing on a shared machine outside the envt

Topology 4 - Multiple ATs and DTs, load balancers and environments joined to a different domain

DNS conditional forwarding

Page 28: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Visual Studio ALM + Team Foundation Server BlogVisual Studio Team

Installing TFS 2012 with Lab Management 2012 BlogBy Martin Hinshelwood

Visual Studio Lab Management videosLab Management Team

Visual Studio Lab Management GuideBy the Visual Studio Rangers

Other resources

Page 29: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Lab Management Summary

Virtual Machines

Hyper-V Host

Automated Build-

Deploy-Test Workflow

Reduces Hardware Utilization

Remote Diagnostic Collection

Virtual Environment Setup/Tear

down

Built on Hyper-V

Integrated with TFS 2012

Enabled by SCVMM 2012Integrated management

Multi-hypervisor management

P2V and V2V

Template deployment

Cloning

Optimize resource usage

Managed by Microsoft Test Manager

Page 30: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine
Page 31: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Appendix

Page 32: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Lab Management HighlightsCreate richer actionable bugs

Simplifies environment setup and snapshots

Create checkpoints to aid defect resolution

Quickly debug issues in the same environmentConnect to the test environment snapshot with network isolation

Automate end-to-end workflowOut of the box build-deploy-test automation workflow

Revert to baseline configuration

RequiresVisual Studio Ultimate or Visual Studio Premium

Team Foundation Server

Microsoft Test Manager

IncludesSystem Center Virtual Machine Manager license

Page 33: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

IIS DB

10.10.1.5 10.10.1.6

172.22.48.15 172.22.48.16

Copy 1

IIS DB

10.10.1.5 10.10.1.6

172.22.48.17 172.22.48.18

Copy 2

Internal network

Lab

network

Page 34: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Lab Management Walkthrough

Page 35: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Reviewing a Rich Bug

Detailed test steps including indexed video recording of test session and

exact failed step

A

• Environment snapshot taken and attached

• Screen capture showing exact point of failure

B

Page 36: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

SCVMM 2012

Page 37: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine
Page 38: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Create new multi-machine environment

choices

Start or Shutdown environments

Page 39: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Choose between two environment types

Page 40: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Configure machine to run UI tests

Page 41: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Create a multi-machine environment as close to production

as possible for testing

Page 42: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine
Page 43: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Select the special lab build

template

Change the lab wizard

workflow parameters

Select an environment to

deploy a build to

Page 44: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Queue a new build or

deploy an existing one

Page 45: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Specify the deployment options to

move application to servers

Select the Test Plans and Test

Suites to run in environment

Page 46: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Test against Lab

Environments Automatically

Auto deployment of current

build

Workflow run for

Deployment and Lab

Page 47: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Thumbnail view of each machine in

the environment

Test Agent automatically started

Page 48: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Environment snapshot taken

and attached as a link with bug

Developers can quickly debug

issues on the exact same testing

environment

Page 49: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Storing in Library

Select environment to store

Page 50: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Easily deploy currently stored

environments

Names of all the environments

stored in the library

Page 51: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Right-click to connect to an

environment

Connect to individual machines via

the environment viewer

Page 52: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

See each machine in

environment

Shows desktop on individual

machine via the environment

viewer

Page 53: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Complete Snapshot

maintenance across ALL

machines in environment

Shows test execution in

environment

Page 54: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

Support for GEO-Distributed Labs

Host Group-US

TFS App Tier

Library Share-US

Library Share-India

Host Group-India

Host MachineHost Machine

Host Machine

US Lab - Redmond

US Lab - Hyderabad

Library Server

Host MachineHost Machine

Host Machine

Library Server

Page 55: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

SCVMM Environments Standard Environments

Testing

Run manual tests

Run coded UI and other automated tests

File rich bugs using diagnostic adapters

Build deployment

Automatic build-deploy-test workflows

Environment creation and management

Use physical machines in addition to virtual machines

Use third-party virtual machines

Automatically install test agents

Save and deploy lab environment snapshots

Create lab environments from VM templates

Start/stop/snapshot environment

Connect to the environment using Environment Viewer

Run multiple copies of an environment using network isolation

Environment Differences

Page 56: Visual Studio Lab Management · PDF fileManual testing Automated testing. ... Visual Studio Lab Management Guide ... US Lab - Redmond US Lab - Hyderabad Library Server Host Machine

REQUIREMENTS

MonitorSprint

WORKING SOFTWARE

DefineIdeation

DevelopIdea to working software

• PowerPoint Storyboarding

• Agile Planning Tools

• Sprint, Backlog, Portfolio

• Code Lens

• Team Rooms

• Suspend & Resume

• Code Reviews

• Feedback Collection

• Unit Testing

• Exploratory Testing

• Continuous Integrations

• Continuous Deployments

• SCOM Integration

• IntelliTrace in Production

• PreEmptive Analytics

• Understand the Domain

• Understand Code Complexity

• Maintain Control

OperateDeployment to feedback

ArchitectArchitectural Discovery

MeasureActionable learning

• Continuous Integration

Continuous Value Delivery