Database continuous integration, unit test and functional test
Post on 23-Jun-2015
282 Views
Preview:
DESCRIPTION
Transcript
Database Continuous Integration, Unit Test and Functional Test
About myself:
• Data architect/DBA with 16 years of SQL Server experience
• Independent consultant, currently Architecture Lead and Technical Delivery Manager at Government of Alberta
• Microsoft Certified System Engineer: MCSE
• Oracle Certified Professional: Oracle DBA
• IBM Certified Solution Expert: DB2 UDB
http://netdbsolutions.comhttps://twitter.com/HarryZheng
http://ca.linkedin.com/in/harryzhenghttps://www.facebook.com/Harry.H.Zheng
2
Session agenda
1. What is Continuous Integration?
2. Database Continuous Integration Setup
3. Database unit test and functional test
4. Database test standard and best practice
5. Tips & Tricks on writing database tests
6. Q&A
3
What is Continuous Integration
• Continuous integration (CI) – the process of continuously integrating developers code in order to find problems quickly
• Wikipedia: CI is the practice of merging all developer working copies with a shared mainline several times a day.
• CI is often related to Test Driven Development (TDD) and Agile practice
4
Traditional work flow
1. Dev A works on his computer and checks in code when ready
2. Dev B works on his computer and checks in code when his task is done
3. Dev C…4. When time to release, we start up build
process and see if the combined code works
5. The version of code doesn’t build6. Who is responsible?
5
Continuous Integration Work Flow
6
What does CI check?
• Does the solution compile?• Does the code pass our unit tests?• Does the code meet functional
requirements?• Does the version meet code quality
requirements?
7
What does CI produce?
• Automatically produce build artifacts• Ensure same version can be staged to
various environments• Setup for automated deployment
8
Benefits of CI
• Ensure code in source control works• Less time on integration• Reduce friction within project team• Save time on build and deploy• Increased visibility of progress• Increased overall confidence on code
9
Do you need CI: scenario 1
• Harry works with 3 other Devs on a side project• Project is a web based app with SQL backend• Harry is the only dba on the team• SQL database is relatively simple with 30 tables and a few
SPs• Do we need setup CI?
10
Do you need CI: scenario 2
• Harry works in a team with 20 developers, 4 DBAs• Main SQL Server has 200+ tables and many SPs, functions• Some developers have good knowledge of SQL skills, others
not so good but can code T-SQL• We used to have a rule that only DBAs can check in SQL
code, but DBAs became a bottom neck quickly• Now we have opened database projects to developers to
check in code changes, bug fixes, etc.• Do we need setup CI?
11
Software for Running CI
• Team City• Cruise Control• Final Builder
12
TeamCity Web Interface
13
Final Builder Server Web Interface
14
Session agenda
1. What is Continuous Integration?
2. Database Continuous Integration Setup
3. Database unit test and functional test
4. Database test standard and best practice
5. Tips & Tricks on writing database tests
6. Q&A
15
Prerequisite for Continuous Integration
• Source Control System• Build Server• Manage Database project with SSDT• Database Tests
16
Database CI Work flow
1. Developer check in code to source control2. Check in triggers a build of the project3. On successful build, the version of the code
is deployed to testing databases4. Unit tests are executed against test
database5. Functional tests are executed against
functional test database6. If all steps succeed, the version is ready for
deployment7. If any step fails, alert is sent to the team
and the cycle starts from step1 again17
Database CI
18
Database deploy script
19
Substitution/Build configuration file<Environment value="BuildDB_CI" machine="VM-DBServer-1" substitute="false"> <Database environmentsuffix="_CI" BackupFileFolder="\\VM-DBServer-1\Deploy"> <SQLInstances> <SQLInstance name="SQLInst1" host="VM-DBServer-1" sqlinstance="\Dev"
databaseFolder="d:\SQLData\Dev" databaseLogFolder="d:\SQLLog\Dev" dbdeployfolder="D:\Deploy"> <Projects> <Project name="DBProj1" seedserver="VM-DBServer-1\Seed" seeddatabase="DB1_Seed" fullbuild="No"> <Databases> <Database name="DBProj1" targetdatabase="DB1_CI" /> </Databases> </Project> <Project name="DBProj2" seedserver="VM-DBServer-1\Seed" seeddatabase="DB2_Seed" fullbuild="No"> <Databases> <Database name="DBProj2" targetdatabase="DB2_CI" /> </Databases> </Project> </Projects> </SQLInstance> </SQLInstances> </Database> <SSIS ServerName="VM-DBServer-1" /> <SSRS ServerName="VM-DBServer-1" ReportServerPath="ReportServer_Dev"/></Environment>
20
Deploy DB schema with SQLPackage
"C:\Program Files (x86)\Microsoft SQL Server\110\DAC\bin\SqlPackage.exe" /Action:Publish /SourceFile:"%1" /TargetConnectionString:"Server=%2;Initial Catalog=%3;Integrated Security=True" /p:BlockOnPossibleDataLoss=False /p:CreateNewDatabase=False /p:DisableAndReenableDdlTriggers=True /p:DropConstraintsNotInSource=True /p:DropDmlTriggersNotInSource=True
21
Deploy database and preserve data
1. Run Pre-deployment scripts2. Execute sqlpackager.exe to deploy
database schema changes3. Run Post-deployment scripts
22
Deploy database and preserve dataWhen to use Pre and Post scripts?•If you change table definition that may cause data loss when database project is deployed, you need to write a pre-deployment script to preserve the data•All other data change scripts will go to Post-deployment script•All Pre and Post scripts should be created so that the scripts can be executed repeatedly
23
Session agenda
1. What is Continuous Integration?
2. Database Continuous Integration Setup
3. Database unit test and functional test
4. Database test standard and best practice
5. Tips & Tricks on writing database tests
6. Q&A
24
What is unit test?
• Test on a discrete unit of code• Unit test should not be affected by other
units• Isolates the unit under test from the rest
of the code• Repeatable
25
My definition of database unit test
• Test that runs quickly, and does not rely on existing data with the exception of static data (e.g.., reference data)
26
Unit Testing - AAA
•Arrange• Setup data (with TestHelper)
•Act• Call Precedure/Function
•Assert• Compare results with expectation
27
Create Unit Test From SSOE from VS
28
Unit Test Example for SP
CREATE PROCEDURE [dbo].[uspShowMeetingAttendee]@MeetingID int = 0
ASSELECT MT.Name,M.FirstName,M.[LastName]From Meeting MTinner join MeetingAttendee MAon MT.MeetingID = MA.MeetingIDinner join Member Mon M.MemberID = MA.MemberID
RETURN 0
29
Unit Test Example-- database unit test for dbo.uspShowMeetingAttendeeBEGIN TRANSACTION
--ArrangeDeclare @MeetingID int
INSERT INTO [dbo].[Meeting] ([Name]) VALUES ('XXTestMeeting')
select @MeetingID = MeetingID from Meeting where Name = 'XXTestMeeting'
INSERT INTO [dbo].[Member]([FirstName],[LastName],[BirthDate]) VALUES ('XXFirstName','XXLastName','1970-01-01')
INSERT INTO [dbo].[MeetingAttendee] ([MeetingID],[MemberID]) Select MeetingID, MemberID From Meeting M1 inner Join Member M2 on 1=1 where M1.Name = 'XXTestMeeting' and M2.LastName = 'XXLastName'
--code to create a dynamic temporary table ##xx{Schema}{SPName}xx -- based of the output of a stored procedure exec [TestHelper].[uspCreateTableForStoredProcedureOutput]@storedProcedure = '[dbo].[uspShowMeetingAttendee]'
--ACT--insert into the temporary table created by the SPinsert into ##xxdbouspShowMeetingAttendeexxEXECUTE [dbo].[uspShowMeetingAttendee] @MeetingID = @MeetingID;
--ASSERT--now you can select specific column without knowledge of the oridinal valueselect LastNamefrom ##xxdbouspShowMeetingAttendeexx where FirstName = 'XXFirstName'
ROLLBACK TRANSACTION
30
What is functional test?
• Functional testing is a type of black box testing that bases its test cases on the specifications of the software component under test
• Functions are tested by feeding them input and examining the output, and internal program structure is rarely considered
31
My definition of database functional test
• Tests with long execution times and they rely on existing non-static data
• Functional tests will run on a server where the complete dataset is available.
32
Database Test Tools and Frameworks
• Microsoft SSDT • http://msdn.microsoft.com/en-ca/data/tools.aspx
• TSQLT • http://tsqlt.org/
• Red-Gate SQL Test• http://www.red-gate.com/products/sql-development/sql-test/
33
Session agenda
1. What is Continuous Integration?
2. Database Continuous Integration Setup
3. Database unit test and functional test
4. Database test standard and best practice
5. Tips & Tricks on writing database tests
6. Q&A
34
Database test standard and best practice
• The folder structure within the database test project will have a structure that closely resembles the structure of the database project.
35
Naming Conventions
Database tests must be discoverable. All database tests will follow one of these naming conventions: <Database Object Name>_Main_<Description>.cs<Database Object Name>_State_<Description>.cs<Database Object Name>_Defect_<Defect #>.cs36
Rules
• All database objects must have at least one main test.
• Static tables (e.g., reference data) must have one or more state tests.
• A defect that results in a change to a database object must have at least one defect test.
37
Rules
• All test scripts must not change the state of the database.
• This is done by adding a Begin Tran/ Rollback Tran code block to the test script.
38
Test Helpers
• In order to assist in creating database tests, the database has a number of stored procedures to create test data.
• All of these stored procedures exist in the [TestHelper] schema.
• Developers are free to modify existing [TestHelper] stored procedures provided the changes do not alter the original purpose of the stored procedure and the changes do not impact any existing database tests.
39
Documenting Tests
All test scripts should be well documented. 1.The comment will indicate which test condition(s) are associated with a particular result set.
2.The comments will indicate what part of the SQL in the database object is being exercised (e.g., which case statements within a where clause are exercised).
40
Session agenda
1. What is Continuous Integration?
2. Database Continuous Integration Setup
3. Database unit test and functional test
4. Database test standard and best practice
5. Tips & Tricks on writing database tests
6. Q&A
41
Writing tests for SP or Function
Problem:•A test that calls to the stored procedure returns a result set with many rows and the test involves retrieving one row only
•A test is only concerned about testing a single value, but the stored procedure returns a result set with many columns
42
Writing tests for SP or Function
There are a couple of ways of making the tests less susceptible to changes: 1.Using a dynamic temporary table2.Using a declared temporary table The first is fair more desirable but cannot be used for stored procedure or functions that have dynamic SQL.
43
Insert into a dynamic temporary table
This approach uses sys.dm_exec_decribe_first_result_set_for_object to dynamically create a temporary table that has the same columns as the first result set of the stored procedure.
[TestHelper].[uspCreateTableForStoredProcedureColumns] creates a global temporary table that can be used to store the results of executing the stored procedure under test.
44
Insert into a dynamic temporary table
Begin Tran
--code to create a dynamic temporary table ##xx{Schema}{SPName}xx
-- based of the output of a stored procedure exec [TestHelper].[uspCreateTableForStoredProcedureColumns]
@storedProcedure = '[dbo].[uspRetrieveUserInfo]'
--insert into the temporary table created by the SPinsert into ##xxdbouspRetrieveUserInfoxx
exec [dbo].[uspRetrieveUserInfo] @UserID = 51
--now you can select specific column without knowledge of the oridinal value
select FirstName from ##xxdbouspRetrieveUserInfoxx where [BirthDate] > '1995-01-01'
Rollback Tran
45
Insert into a dynamic temporary table Create procedure [TestHelper].[uspCreateTableForStoredProcedureOutput] ( @storedProcedure varchar(128))asbegin declare @sql nvarchar(max) = '', @tablename varchar(100) = '##xx' + replace(replace(replace(@storedProcedure,'[',''),']',''),'.','') + 'xx' set @sql = @sql + 'IF OBJECT_ID(''tempdb..' + @tablename + '''' + ') IS NOT NULL DROP TABLE ' + @tablename + ';' set @sql = @sql + 'create table ' + @tablename + '(' select @sql = @sql + '[' + name + '] ' + coalesce(system_type_name, (D.Data_Type +
case when Character_Maximum_Length is not null then '(' + cast(Character_Maximum_Length as varchar) + ')'
else '' End)) + ',' from sys.dm_exec_describe_first_result_set_for_object (object_id(@storedProcedure), null) FR Left Outer Join INFORMATION_SCHEMA.DOMAINS D on FR.User_Type_Name = D.Domain_Name Order by FR.Column_Ordinal set @sql = substring(@sql, 0, len(@sql)) + ');' EXEC sp_ExecuteSQL @sql;end
46
Insert into a dynamic temporary table Pros:• Test is not dependent on the order of the
columns in the stored procedure, i.e., the order can change
• Result set can be further filteredCons:• Will not work for stored procedures that
use dynamic SQL • Will not work for stored procedures that
return more that one result set
47
Insert into a declared temporary table
Declare a temporary table that has the same columns as the result set of the stored procedure.
This is the less desirable of the two options but is the only choice if the stored procedure or function returns the result set using dynamic SQL.
48
Insert into a declared temporary table Begin Tran
-- TestHelper SP creates a simple user with a first name of xxFoobarxxdeclare @UserID Intexec [TestHelper].[uspCreateUser] @id = @UserID output
declare @User table ( ID Int, FirstName varchar(50),LastName varchar(50),BirthDate date)
--Insert the SP output to the temp tableinsert into @User exec dbo.uspSearchUserByName @FirstName = 'xxFoobarxx‘
--this should return a single user--Verify_One_User_Existsselect * from @User where FirstName = 'xxFoobarxx‘
Rollback Tran
49
Insert into a declared temporary tablePros:• Further filtering of the result set is
possible.
Cons:• The order of the columns in the declared
temporary table must be the same as the order of the result set returned by the stored procedure or function.
50
Session agenda
1. What is Continuous Integration?
2. Database Continuous Integration Setup
3. Database unit test and functional test
4. Database test standard and best practice
5. Tips & Tricks on writing database tests
6. Q&A
51
52
Reference and further reading:
1. http://www.pluralsight.com/courses/continuous-integration2. http://en.wikipedia.org/wiki/Continuous_integration3. http://www.rackspace.com/blog/the-business-advantages-
of-continuous-integration/4. http://www.youtube.com/watch?v=PbVKgoAVZjY
(Continuous Integration with the Database by Ike Ellis)5. http://indragunawan.com/2013/09/teamcity-continuous-
integration-for-everybody.html6. http://www.youtube.com/watch?v=4sANX9AhM8c
(Introduction to Continuous Integration)
53
top related