Transcript
Stub-O-Matic DocumentationRelease 0X beta
OpenCredo
February 09 2017
Contents
1 Introduction 111 Background 112 Applicability 1
2 Stub Basics 321 Load and Retrieve Stubs 322 Sessions and Scenarios 423 Recording Stubs 4
3 Stubo Command File 731 Command file (YAML) 732 Command Scripting 9
4 Stubo Command File (text format) 1141 Text Command file 1142 Command Scripting 1243 Passing Arguments 12
5 GUI 1351 Tracker 1352 Manage 1353 Analytics 14
6 Integration with Test Tools 1561 Test steps example 15
7 Stubbing 1771 Add Delay to a Response 1772 Matching 1773 Body contains matching 1874 Templated Matcher 1875 Templated Responses 1876 Request Data in Responses 1977 Stateful Stubs 2078 User Exits 2079 Hooks 21710 Modules 21711 Splitting 21712 Caching Values 21
i
8 Sharing Stubs 23
9 Communicating with Stub-O-Matic 2791 GetResponse URLs 2792 Virtual Stub-O-Matic 27
10 29
11 31
12 33
13 35
14 37
15 39
16 41
17 43
18 45
19 47
20 49
21 Stub-O-Matic API v1 51211 execcmds 51212 getversion 52213 getstatus 53214 beginsession 54215 endsession 54216 endsessions 55217 putscenarios 55218 putstub 55219 getstublist 562110 putdelay_policy 572111 getdelay_policy 582112 deletedelay_policy 582113 getresponse 582114 deletestubs 592115 getexport 592116 getstubcount 612117 putmodule 612118 getmodulelist 622119 deletemodule 622120 deletemodules 622121 Set Tracking Level 632122 Blacklist a host URL 642123 Create Bookmark 642124 Jump to Bookmark 652125 Delete Bookmark 652126 List Bookmarks 652127 getstats 65
ii
22 Examples 67221 Templates Date Rolling 67222 Stateful stubs 67223 Auto Mangling with user exits 67224 user exit using XSLT 68225 User exit using a cache 68
23 Clients (Stubo Integrator) 69231 Test data Setup and Teardown 69232 To Stub or not to Stub 70233 Playback of Stubs 71234 Record Stubs 71235 Performance Requirements 72236 Error Responses 72
24 Reference 73241 Stub Definition 73
25 About 75
26 Change History 77261 stubo Change History 77
27 Indices and tables 79
iii
iv
CHAPTER 1
Introduction
The target of the Stub-O-matic software is to enable automated testing by mastering system dependencies
11 Background
Stub-O-Matic (sometimes called Stubo) is a service which provides data stubs for software testing Stubs enable thesoftware being tested to be isolated from the rest of the environment and 3rd party back-end systems enabling repeatedautomated testing Stubo is capable of supporting functional load and performance testing
Stubbing enables development to progress when the actual dependencies are not available
Stub-O-Matic stubs include just enough intelligence to emulate back-end systems without the complexity often asso-ciated with data set-up activities
The resilience of testing and automated testing is improved with stubs because it is possible to manage stubs moreeasily than 3rd party systems and databases Dependent systems are often not capable of creating the full range of testconditions necessary to fully test code Refer to httpenwikipediaorgwikiTest_stubs for further information
12 Applicability
Use Stub-O-Matic when you donrsquot have as much control over your associated test data to achieve the level of testingrequired Stub-O-Matic utilises the ubiquitous HTTP protocol to receive stub requests and return stubbed responses tothe system under test To use stubs the system under test must be capable of diverting a request that would normallygo to back-end systems How this is best done depends on the architecture of the system under test Implementationsto date have used JAVA AOP Python and ActionalProgress Stub Itineraries
Stubo supports unicode It has not been designed for requests or responses containing bit maps or other non-standardcharacters
1
Stub-O-Matic Documentation Release 0X beta
2 Chapter 1 Introduction
CHAPTER 2
Stub Basics
Stubo simulates back-end systems by matching a request to a response RequestResponse definitions are lsquoloadedrsquo intoStubo
Conventions for this documentation
All Stubo commands are to be prefixed with the server (and port) of the Stubo instance you are using For examplestuboapigetstatus would really be something like lsquohttpmystubostuboapigetstatuslsquo
21 Load and Retrieve Stubs
For the impatient point your browser at stubodefaultexeccmdscmdfile=staticcmdsdemofirstyaml The JSONresponse from Stubo is meant for machines to read For a more human friendly version use the lsquobackrsquo button on yourbrowser and navigate to the lsquoTrackerrsquo page There you will see the most recent Stubo commands and their details Thecommands you just ran from firstyaml are
First use of Stub-O-Matic run this from a browser with uri httpltstubo servergtstubodefaultexeccmdscmdfile=staticcmdsdemofirstyaml
Describe your stubs hererecording
scenario firstsession first_1stubs-json
request method GETbodyPatterns [
contains [get my stub]
]
response
status 200body Hello 1+1 World
3
Stub-O-Matic Documentation Release 0X beta
Provide your requests hereplayback
scenario firstsession first_1requests-
json method GETbody timestamp 092345
get my stub
These will be listed on the Tracker page What just happened The command you made executed a Stub-O-Maticcommand file (demofirstyaml) That file loaded a stub into Stuborsquos database then simulated a request returning theexpected response These actions were logged and displayed on the Tracker page
There are a few key concepts that will make stubbing work
bull Scenario - A re-usable set of stubs
bull Session - The instance or use of the stubs from a scenario
bull Matcher - The matcher used to find the correct response for a request
bull Request - A message normally sent to a back-end by the system under test
bull Response - A message normally returned by a back-end system
bull Stub = Matcher(s) and a response
Stubo commands have been designed to be read written and used by either humans or the Stubo code The stub andcommand files are accessed via a URI (http) In practice this means putting them into a source code repositorysuch as Subversion (SVN) which you should be doing anyway The file location in the execcmds command can belocal to Stubo as in the firstyaml example or (more usual) a URL
For example you could run
stubodefaultexeccmdscmdfile=httpsyour-source-code-repostaticcmdsdemofirstyaml
22 Sessions and Scenarios
In order for stubs to be loaded and retrieved in a predictable automated manner by multiple users many Stubo eventsare controlled by a session A session is a unit of work that must be started and ended Sessions have two possiblemodes of operation
bull record - for loading stubs into the Stubo datastore
bull playback - for retrieving stubbed responses during testing
A scenario is a group of stubs that can be used and re-used by multiple sessions
23 Recording Stubs
Stubs can be created by hand or auto-recorded from a real system under test This is done from what we call a lsquoStubointegrator or Stubo client servicersquo A Stubo integrator is typically a custom library that intercepts real system calls andlsquore-playsrsquo them into a Stubo lsquorecordingrsquo via putstub commands Java and Python Stubo client libraries are provided
4 Chapter 2 Stub Basics
Stub-O-Matic Documentation Release 0X beta
to help integrate Stubo into your applications These clients are maintained in github at httpsgithubcomStub-O-Maticpython-client amp httpsgithubcomStub-O-Maticjava-client
If the real system changes its interface a recording can be repeated to capture the changes Of course this requires thatthe test back-end systems have the correct data in them to be used successfully
Stubo provides mechanisms to deal with context sensitive information such as transaction ids dates user names etcwhich makes recorded stubs less fragile
23 Recording Stubs 5
Stub-O-Matic Documentation Release 0X beta
6 Chapter 2 Stub Basics
CHAPTER 3
Stubo Command File
Command files are used to load stub files and add behaviour to them (state dates etc)
Stubs can also be loaded and tested using individual calls to the Stub-O-Matic server using the Stubo HTTP APICommand files are a shorthand for the API making it simpler to enable repeatable automated interactions with Stub-O-Matic by grouping a set of commands together required for a particular test
31 Command file (YAML)
An example
Stubo YAML Execution ordering is 1) commands 2) recording 3) playback
Commands go herecommands
-putmodule
varsname staticcmdstestsrestyamlnooppy
-putdelay_policy
varsname slowdelay_type fixedmilliseconds 1000
Describe your stubs hererecording
scenario restsession rest_recordingstubs-file stub1jsonvars
recorded_at as_date(2015-01-10)ext_module noop
-json
request method GETbodyPatterns [
7
Stub-O-Matic Documentation Release 0X beta
jsonpath [cmdx]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all ok 2headers Content-Type applicationjsonX-Custom-Header 1234
vars
foo bar
Provide your requests hereplayback
scenario restsession rest_playbackrequests-
file request1jsonvars
played_at as_date(2015-01-20)-
json method GETbody cmd x yheaders
Content-Type applicationjsonX-Custom-Header 1234
varsplayed_at as_date(2015-01-20)
Note the payloads can be defined either in place with the lsquojsonrsquo key or point to an external reference via the lsquofilersquo keyThe file can either reference a local statically served file as shown above or be a URI reference Typically it will be aURI reference to a location in a users source control system where their stubs are stored
The local file references in this example are shown below
stub1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
8 Chapter 3 Stubo Command File
Stub-O-Matic Documentation Release 0X beta
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
request1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
Note that these json payloads for the request and response are defined as strings Stubo also excepts the definition asdictionaries
32 Command Scripting
The YAML file is run through a template processor before being parsed and executed by Stubo Any variables definedsuch as lsquoplayed_atrsquo will evaluated and appropriate subsitutions made
A roll date example
(daterollyaml)
playbackrequests- file dateroll_1433930288_0requestresponse dateroll_1433930288_0stubo_responsevars
getresponse_arg this stub was played at 2015-06-10 095744839438
32 Command Scripting 9
Stub-O-Matic Documentation Release 0X beta
play_date 2014-09-12priority 1putstub_arg this stub was recorded at 2015-06-10 095744839387rec_date 2014-09-10tracking_level full
scenario daterollsession dateroll_1433930288
recordingscenario daterollsession dateroll_1433930288stubs- file dateroll_1433930288_0json
Referenced files
dateroll_1433930288_0json
priority 1args
priority 1rec_date 2014-09-10putstub_arg this stub was recorded at 2015-06-10 095744839387
request
bodyPatterns contains [
ltxml version=10 encoding=UTF-8 standalone=yes gtnltrollmegt n ltOriginDateTimegtroll_date(2014-09-10 as_date(rec_date) as_date(play_date))T000000ZltOriginDateTimegtnltrollmegt]
method POST
response
body ltresponsegtnltputstub_arggt raw putstub_arg ltputstub_arggtnltgetresponse_arggt getresponse_arg ltgetresponse_arggtnltresponsegtstatus 200
dateroll_1433930288_0request
body ltxml version=10 encoding=UTF-8 standalone=yesgtnltrollmegtn ltOriginDateTimegt2014-09-12T000000ZltOriginDateTimegtnltrollmegtheaders host nullpath nullquery uri nullmethod POST
10 Chapter 3 Stubo Command File
CHAPTER 4
Stubo Command File (text format)
The line orientated text version of the command file is an old format It does not support the matching required forREST For example you canrsquot match on the request headers or method It only supports one type of matcher whichis the lsquobody containsrsquo matcher If the text specified for the lsquobody containsrsquo matcher stub is found in the request amatch is found This is often however the only type of matcher required to successfully stub a system This format isstill supported for import via execcmds but will not be exported anymore If a text commands file is imported via alsquocommandsrsquo text file on export via getexport a lsquoyamlrsquo file will be exported
41 Text Command file
bull Lines beginning with a lsquorsquo are treated as comments
bull Commands follow the Stub-O-Matic REST API for example be-ginsessionscenario=abcampsession=12345ampmode=record
bull When the API requires something in the HTTP body input can be sourced by listing files after the command(separated by commas) Files are assumed to be in the same directory as the command file Alternatively a fullURI can be provided to source the input (matcher request or response)
bull Blank lines are ignored
bull An example command is putstubsession=12345rsquofeng_001textMatcher feng_002textMatcherfeng_001response A putstub command must have at least one matcher file and one response file Thisexample has two matchers
What goes in the command files
bull A textMatcher file could be some or all of the text from the request eg AB1234ltFlightInfoQuerygt
bull Multiple textMatchers files may be used in a stub All must evaluate to True against the request for a responseto be returned
bull request files contain the text of a request (eg text xml json) Note the using playback in commands is onlyused to test the stubs as these would be issued by the real system being tested in normal situations
bull response files contain the text of a response (eg text xml json)
bull Any naming convention can be used for matcher request and response files We have used suffixes oftextMatcher request amp response but you are free to use other extensions like matcherxml requestjson re-sponsexml etc
11
Stub-O-Matic Documentation Release 0X beta
42 Command Scripting
Stub-O-Matic supports the ability to load stubs from various and multiple sources This means people can organise allthe stubs that support a particular service or use case together and any test that needs those stubs could pull them infrom common stub libraries maximising stub reuse
set foreign_url = httpwwwgooglecouk set other_url = httpwwwsuncom deletestubsscenario=smartbeginsessionscenario=smartampsession=smart_1ampmode=recordputstubsession=smart_1url=other_urlurl=foreign_urlputstubsession=smart_1text=random_rubbletext=response_textendsessionsession=smart_1
Command files are also programmable with Python code snippets See the example below
set session_name = smart_1 deletestubsscenario=smartbeginsessionscenario=smartampsession=session_nameampmode=recordputstubsession=session_namefirsttextMatcherfirstresponseendsessionsession=session_namebeginsessionscenario=smartampsession=session_nameampmode=playback make 3 requests for i in range(03) getresponsesession=session_namefirstrequest end endsessionsession=session_name
43 Passing Arguments
Arguments can be passed into command files and used in them The following example allows the session and scenarioto be named externally to the command file
execcmdscmdFile=staticcmdstestsacceptsmart_argcommandsampscen=bobampsession=smart_1
The above call can be used as follows
deletestubsscenario=scen[0]beginsessionscenario=scen[0]ampsession= session[0] ampmode=recordputstubsession=session[0]firsttextMatcherfirstresponseendsessionsession=session[0]beginsessionscenario=scen[0]ampsession=session[0]ampmode=playback for i in range(03) getresponsesession=session[0]firstrequest end endsessionsession=session[0]
12 Chapter 4 Stubo Command File (text format)
CHAPTER 5
GUI
51 Tracker
The Stubo lsquoTrackerrsquo page shows details of all requests submitted to Stub-O-Matic and the received response Clickon the link in the function column to show a detailed view Debug level tracking can be added to a getresponse callif lsquotracking_level=fullrsquo is added to the call Matching errors will also trigger some extra debug data to be added to anormal track record
Stubo can not keep tracker data forever so it will eventually drop off the server as space is used up
The tracker page shows all Stubo actions performed on the server Filters are provided to filter on session datetimeresponse times and errors
To go into detail on a particular transaction click the link in the lsquoFunctionrsquo column or lsquoDetailsrsquo in the lsquoStubo Responsersquocolumn For a getresponse call the matchers and responses for the call can be found from the gtgtgt view scenario details(stub matchers and responses) link at the top of the details page
Filtering The tracker filter fields should be self explanatory By default the tracker page shows only data sent to theStubo URL shown at the top of the browser To see tracker items from other URLs on the same hardware tick thelsquoAll Hostsrsquo box
52 Manage
The manage page allows users to execute command files and view system state
The list of loaded stubs includes the date the stubs were loaded into Stubo and the last time the stub was used Clickingthe scenario name shows the stored matchers and responses for the scenario
Also available is a command file scratchpad for entering one or more commands For example one could enter
endsessionsession=first_sessiondeletestubsscenario=firstendsessionsession=second_sessiondeletestubsscenario=second
All the above commands would then be executed See the results on the tracker page
The manage page also shows stubs delay policies and any external modules loaded
13
Stub-O-Matic Documentation Release 0X beta
53 Analytics
The lsquoAnalyticsrsquo tab takes you to a dashboard showing statistics about the Stubo server The orange bar along the topallows you to select different Stubo servers and time periods The first of the 4 squares on the right turns on a legend
Charts
chart title descriptionstubssecondresponse rate
number of stubs requested per second
delay added user defined delays to stubs these can be specified to emulate response times of thesystem being stubbed
latency average time taken to return a stub excluding network and user defined delaysrequestresponse size size in kb of the requests to stubo and responses deliveredhost response latency where Stubo is deployed across multiple servers this chart compares the different servers
against each otherfailure rate percent of failed requests (HTTP response gt= 400)health checks Stubo has a regular health check ping to demonstrate that all systems are working
properlyrunning putstubcount
cumulative total of stubs loaded
running responsecount
cumulative total of stubs retrieved
Also note the Stubo API documentation for getstats
14 Chapter 5 GUI
CHAPTER 6
Integration with Test Tools
Stub-O-Matic works with all major software test execution tools Stubo of course does not run tests or evaluateresults it emulates back-end systems needed by tests
Data dependent software tests typically consist of 3 steps
1 Data setup
2 Test execution
3 Data teardown
Working with Stubo to provide back-end data is no different
Stuborsquos API can be used to setup or teardown data directly Stubo command files offer a shorthand for using the APIand are much more compact Any test tool that can make HTTP calls can use Stubo
61 Test steps example
1 Trigger an http call similar to this from the test tool of your choice
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_setupcommands
2 Run tests
3 Trigger a teardown http call similar to this from the test tool
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_teardowncommands
Setup would typically i) delete stubs ii) set any delays iii) submit a recording via a series of putstub calls and finallyiv) begin a session in playback
Teardown would typically i) end session ii) delete stubs
Setup and teardown can be performed with a mixture of command files (YAML or commands) and direct HTTP APIcalls to stubo
15
Stub-O-Matic Documentation Release 0X beta
16 Chapter 6 Integration with Test Tools
CHAPTER 7
Stubbing
71 Add Delay to a Response
To emulate response times of systems being stubbed one can set a delay on stubs Delays are set on the stub responsenot on the request to stubo This is because one request into the system being tested can result in several back-endstubocalls each of which might need different delay settings
Stubo provides reusable delay_policy objects which can be used across many stubs and across sessions The de-lay_policy can be altered on the fly during a loadperformance test if desired There are currently two delay_policytypes to choose from (1) fixed and (2)random with mean and standard deviation
A fixed delay policy can be created or modified with a commnand such as
putdelay_policyname=a_fixed_delay_policyampdelay_type=fixedampmilliseconds=200
Note that the value for the lsquonamersquo can be set to any string and is used again in the putstub A variable delay can be setup as follows
putdelay_policyname=a_normalvariate_delay_policyampdelay_type=normalvariateampmean=100ampstddev=50
Allowed lsquodelay typersquo values are lsquofixedrsquo and lsquonormalvariatersquo The next step is to use the delay policy when loadingstubs eg
putstubsession=abcampdelay_policy=a_fixed_delay_policy
bull Whenever the stub above is used the named delay policy will be applied
bull Delay policies may be altered on the fly during a test run if desired To change a delay on the fly simply repeatthe putdelay_policy command with a different value Typically this would be done directly by the test toolrather than from a command file
bull Delay policies can be used across multiple sessions This allows for easier alteration of policies where manysessions use the same policy It also means that you should name policies carefully and only alter or sharepolicies you are in control of
72 Matching
Stub-O-Matic currently supports various types of matchers
17
Stub-O-Matic Documentation Release 0X beta
73 Body contains matching
One or more matchers can be defined whereby all matchers must be contained in the request to return a specifiedresponse A typical difficulty with system requests is that they often contain superfluous information such as sessionIDs or time stamps that get in the way of matching a request to the desired response One way Stub-O-Matic can solvethis problem is by the use of lsquomatchersrsquo Matchers are parts of the request that matter to you only what is needed tofind the correct response file
For example a request may include
ltsessionidgt123456ltsessionidgtlttransactionidgt0987654321-9999lttransactionidgt
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Of these 4 lines only the last 2 matter the first 2 lines will differ from one request to the next and be set by the systemmaking the request This request then will be different each time it is made How can one easily match a request likethis to a canned response file without writing code for each requestresponse pair The solution used by Stub-O-Maticis to not store requestresponse file pairs but rather store matchers and the response In the case above the matcherswould be
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Stub-O-Matic contains code which will take a request as input search for the response with the best matchers Everymatcher must be contained in the request to be a match When a match is found the corresponding response is servedback
All matchers must be present in the request to make a match Stub-O-Matic will return the response from the firstmatch if there is more than one possible match Note that whitespace and carriage returns are ignored when matching
Alternatives to removing parts of the request you donrsquot want to match on are
1 using a matcher template to substitute ids or dates etc from the request into the matcher OR
2 using a user exit to run custom code A convenience class XMLManglerExit is provided for XML payloads toignore XMl elements andor attributes from the matching process
74 Templated Matcher
This is an example of a template matcher than extracts values from the request into the matcher The xmltree object isan lxml parsed root element passed into the template
ltrequestgtlttransidgtxmltreexpath(requesttransid)[0]textlttransidgtltdtgt2009-10-24T000000+0000ltdtgtltflightnumbergt0455ltflightnumbergtltrequestgt
75 Templated Responses
A hand crafted Stubo response may contain variables which are substituted in For example a response may contain
set flight_nbr=XX1234 The flight number is flight_nbr
18 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
The resulting response served wil be
The flight number is XX1234
Code may be used for example adding random elements to a response
import random set rnd_nbr = randomrandint(1200) Hello World 1000 + rnd_nbr
The resulting response served will be
Hello World - followed by a number between 1000 and 1200
To put todayrsquos date in a response include the following
today_strformat(dmy)
Stubo responses are run through a Tornado template and any logic or commands allowed in these templates can beused See httpwwwtornadoweborgenstabletemplatehtml for details
76 Request Data in Responses
761 Text Manipulation
It is possible to pull data from the stub request and put it in the response Say you have 200 stubs that are identicalapart from a userid They could be condensed into one stub which uses the userid from the request in the responserequest
ltuseridgtabc123ltuseridgt
matcher
ltuseridgt
response template (using Python)
Hello to request_text[814]
response
Hello to abc123
762 XML Manipulation
Sometimes it is easier to use XPATH to pull data from a request and place it in a response If you need the SessionIdfrom this request
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtJohn1234ltwbsSessionIdgt
Pull the SessionId from the request and use it within the response template
76 Request Data in Responses 19
Stub-O-Matic Documentation Release 0X beta
set namespaces = soapenv httpschemasxmlsoaporgsoapenvelopewbs httpxmlaaacomws200901WBS_Session-20xsd
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtxmltreexpath(soapenvHeaderwbsSessionwbsSessionId
namespaces=namespaces)[0]textltwbsSessionIdgt
Note the xmltree variable is the parsed xml request (as an lxml Root object) made available to the template if therequest is valid xml
Another example - no namespaces in the request Request
ltxml version=10 encoding=UTF-8gtltCompensateCustomersCheck APCWCustom=-1 LocalCurrencyCode=GBP NumberOfAdults=1gt
Response
set currency_code = xmltreexpath(CompensateCustomersCheck)[0]attrib[LocalCurrencyCode] ltresponse LocalCurrencyCode=currency_codegtpay meltresponsegt
77 Stateful Stubs
State is important in simulating back-end systems For example if one was to check-in passenger Bob on a particularflight the back-end would allow this request to succeed only one time and subsequent attempts to check-in the samepassenger on the same flight would be rejected Stubo can emulate this functionality or if desired for repeatable testsStubo can allow Bob to be checked in multiple times on the same flight
Stateful stubbing is enabled automatically If stubs are recorded or loaded in which the same request returns differentresponses Stubo will remember this and internally keep track of which response should come next
If your test is to check-in Bob multiple times simply do not load the stubs representing the check-in rejection andStubo will allow multiple check-ins
78 User Exits
Stub-O-Matic provides hooks into the runtime to execute custom user code Stubo can execute your custom code totransform stub matchers requests andor responses This is a powerful mechanism when your stubs contain dynamicdata or need to change over time This can be useful to perform lsquointelligent stubbingrsquo for example to roll dates Theuser exit API is contained in the stuboextuser_exit module
781 API
782 XML Auto Mangling
If your data is XML a convenience class XMLUserExit is provided to enable auto mangling
20 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
783 API
79 Hooks
710 Modules
Modules are added with the putmodule command and referenced via the ext_module url arg in the putstub call Forexample
putmodulename=staticcmdstestsextxsltmanglerpydeletestubsscenario=mangler_xsltbeginsessionscenario=mangler_xsltampsession=mangler_xslt1ampmode=recordputstubsession=mangler_xslt1ampext_module=mangler firstrequest firstresponseendsessionsession=mangler_xslt1
This example dynamically imports a test mangler from the Stubo server Typically these external python modules thathook into the Stubo runtime would be sourced from source control system using a URL
Note user exit processing is only enabled if the lsquoext_modulersquo variable is set to a pre-loaded module during the putstubcall Each Stub-O-Matic host URL has its own copy of any loaded modules Such modules are available to any testand as such should typically be loaded once per test suite
711 Splitting
Useful to removing random elements from matchers It can happen that the system under test will include seeminglyrandom elements in the request This will cause the next run of the test to not match This problem can be solved byprocessing matchers when loading - during a record A matcher can be split into two with the variable text left outFuture request must match both of the matchers to get the response An example is
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N11111234567806P1ltCommandgtltCryptic_GetScreen_Querygt
Which becomes two matchers
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N1111
and
06P1ltCommandgtltCryptic_GetScreen_Querygt
cutting out the random number lsquo12345678rsquo in the middle
This matcher splitting can be done manually or automatically with a user exit Seestaticcmdstestsextsplitters1firstcommands for and example It uses the rules module at staticcmdstestsextaaapy
Multiple matchers excluding random request elements can be created manually and loaded as such
putstubsession=splitter_record splitter_1textMatcher splitter_2textMatcher splitterresponse
712 Caching Values
If emulating back-end behaviour means writing some code for a particularly tricky behaviour Stubo exposes a simplekey-value cache API to matchers and responses
79 Hooks 21
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
Contents
1 Introduction 111 Background 112 Applicability 1
2 Stub Basics 321 Load and Retrieve Stubs 322 Sessions and Scenarios 423 Recording Stubs 4
3 Stubo Command File 731 Command file (YAML) 732 Command Scripting 9
4 Stubo Command File (text format) 1141 Text Command file 1142 Command Scripting 1243 Passing Arguments 12
5 GUI 1351 Tracker 1352 Manage 1353 Analytics 14
6 Integration with Test Tools 1561 Test steps example 15
7 Stubbing 1771 Add Delay to a Response 1772 Matching 1773 Body contains matching 1874 Templated Matcher 1875 Templated Responses 1876 Request Data in Responses 1977 Stateful Stubs 2078 User Exits 2079 Hooks 21710 Modules 21711 Splitting 21712 Caching Values 21
i
8 Sharing Stubs 23
9 Communicating with Stub-O-Matic 2791 GetResponse URLs 2792 Virtual Stub-O-Matic 27
10 29
11 31
12 33
13 35
14 37
15 39
16 41
17 43
18 45
19 47
20 49
21 Stub-O-Matic API v1 51211 execcmds 51212 getversion 52213 getstatus 53214 beginsession 54215 endsession 54216 endsessions 55217 putscenarios 55218 putstub 55219 getstublist 562110 putdelay_policy 572111 getdelay_policy 582112 deletedelay_policy 582113 getresponse 582114 deletestubs 592115 getexport 592116 getstubcount 612117 putmodule 612118 getmodulelist 622119 deletemodule 622120 deletemodules 622121 Set Tracking Level 632122 Blacklist a host URL 642123 Create Bookmark 642124 Jump to Bookmark 652125 Delete Bookmark 652126 List Bookmarks 652127 getstats 65
ii
22 Examples 67221 Templates Date Rolling 67222 Stateful stubs 67223 Auto Mangling with user exits 67224 user exit using XSLT 68225 User exit using a cache 68
23 Clients (Stubo Integrator) 69231 Test data Setup and Teardown 69232 To Stub or not to Stub 70233 Playback of Stubs 71234 Record Stubs 71235 Performance Requirements 72236 Error Responses 72
24 Reference 73241 Stub Definition 73
25 About 75
26 Change History 77261 stubo Change History 77
27 Indices and tables 79
iii
iv
CHAPTER 1
Introduction
The target of the Stub-O-matic software is to enable automated testing by mastering system dependencies
11 Background
Stub-O-Matic (sometimes called Stubo) is a service which provides data stubs for software testing Stubs enable thesoftware being tested to be isolated from the rest of the environment and 3rd party back-end systems enabling repeatedautomated testing Stubo is capable of supporting functional load and performance testing
Stubbing enables development to progress when the actual dependencies are not available
Stub-O-Matic stubs include just enough intelligence to emulate back-end systems without the complexity often asso-ciated with data set-up activities
The resilience of testing and automated testing is improved with stubs because it is possible to manage stubs moreeasily than 3rd party systems and databases Dependent systems are often not capable of creating the full range of testconditions necessary to fully test code Refer to httpenwikipediaorgwikiTest_stubs for further information
12 Applicability
Use Stub-O-Matic when you donrsquot have as much control over your associated test data to achieve the level of testingrequired Stub-O-Matic utilises the ubiquitous HTTP protocol to receive stub requests and return stubbed responses tothe system under test To use stubs the system under test must be capable of diverting a request that would normallygo to back-end systems How this is best done depends on the architecture of the system under test Implementationsto date have used JAVA AOP Python and ActionalProgress Stub Itineraries
Stubo supports unicode It has not been designed for requests or responses containing bit maps or other non-standardcharacters
1
Stub-O-Matic Documentation Release 0X beta
2 Chapter 1 Introduction
CHAPTER 2
Stub Basics
Stubo simulates back-end systems by matching a request to a response RequestResponse definitions are lsquoloadedrsquo intoStubo
Conventions for this documentation
All Stubo commands are to be prefixed with the server (and port) of the Stubo instance you are using For examplestuboapigetstatus would really be something like lsquohttpmystubostuboapigetstatuslsquo
21 Load and Retrieve Stubs
For the impatient point your browser at stubodefaultexeccmdscmdfile=staticcmdsdemofirstyaml The JSONresponse from Stubo is meant for machines to read For a more human friendly version use the lsquobackrsquo button on yourbrowser and navigate to the lsquoTrackerrsquo page There you will see the most recent Stubo commands and their details Thecommands you just ran from firstyaml are
First use of Stub-O-Matic run this from a browser with uri httpltstubo servergtstubodefaultexeccmdscmdfile=staticcmdsdemofirstyaml
Describe your stubs hererecording
scenario firstsession first_1stubs-json
request method GETbodyPatterns [
contains [get my stub]
]
response
status 200body Hello 1+1 World
3
Stub-O-Matic Documentation Release 0X beta
Provide your requests hereplayback
scenario firstsession first_1requests-
json method GETbody timestamp 092345
get my stub
These will be listed on the Tracker page What just happened The command you made executed a Stub-O-Maticcommand file (demofirstyaml) That file loaded a stub into Stuborsquos database then simulated a request returning theexpected response These actions were logged and displayed on the Tracker page
There are a few key concepts that will make stubbing work
bull Scenario - A re-usable set of stubs
bull Session - The instance or use of the stubs from a scenario
bull Matcher - The matcher used to find the correct response for a request
bull Request - A message normally sent to a back-end by the system under test
bull Response - A message normally returned by a back-end system
bull Stub = Matcher(s) and a response
Stubo commands have been designed to be read written and used by either humans or the Stubo code The stub andcommand files are accessed via a URI (http) In practice this means putting them into a source code repositorysuch as Subversion (SVN) which you should be doing anyway The file location in the execcmds command can belocal to Stubo as in the firstyaml example or (more usual) a URL
For example you could run
stubodefaultexeccmdscmdfile=httpsyour-source-code-repostaticcmdsdemofirstyaml
22 Sessions and Scenarios
In order for stubs to be loaded and retrieved in a predictable automated manner by multiple users many Stubo eventsare controlled by a session A session is a unit of work that must be started and ended Sessions have two possiblemodes of operation
bull record - for loading stubs into the Stubo datastore
bull playback - for retrieving stubbed responses during testing
A scenario is a group of stubs that can be used and re-used by multiple sessions
23 Recording Stubs
Stubs can be created by hand or auto-recorded from a real system under test This is done from what we call a lsquoStubointegrator or Stubo client servicersquo A Stubo integrator is typically a custom library that intercepts real system calls andlsquore-playsrsquo them into a Stubo lsquorecordingrsquo via putstub commands Java and Python Stubo client libraries are provided
4 Chapter 2 Stub Basics
Stub-O-Matic Documentation Release 0X beta
to help integrate Stubo into your applications These clients are maintained in github at httpsgithubcomStub-O-Maticpython-client amp httpsgithubcomStub-O-Maticjava-client
If the real system changes its interface a recording can be repeated to capture the changes Of course this requires thatthe test back-end systems have the correct data in them to be used successfully
Stubo provides mechanisms to deal with context sensitive information such as transaction ids dates user names etcwhich makes recorded stubs less fragile
23 Recording Stubs 5
Stub-O-Matic Documentation Release 0X beta
6 Chapter 2 Stub Basics
CHAPTER 3
Stubo Command File
Command files are used to load stub files and add behaviour to them (state dates etc)
Stubs can also be loaded and tested using individual calls to the Stub-O-Matic server using the Stubo HTTP APICommand files are a shorthand for the API making it simpler to enable repeatable automated interactions with Stub-O-Matic by grouping a set of commands together required for a particular test
31 Command file (YAML)
An example
Stubo YAML Execution ordering is 1) commands 2) recording 3) playback
Commands go herecommands
-putmodule
varsname staticcmdstestsrestyamlnooppy
-putdelay_policy
varsname slowdelay_type fixedmilliseconds 1000
Describe your stubs hererecording
scenario restsession rest_recordingstubs-file stub1jsonvars
recorded_at as_date(2015-01-10)ext_module noop
-json
request method GETbodyPatterns [
7
Stub-O-Matic Documentation Release 0X beta
jsonpath [cmdx]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all ok 2headers Content-Type applicationjsonX-Custom-Header 1234
vars
foo bar
Provide your requests hereplayback
scenario restsession rest_playbackrequests-
file request1jsonvars
played_at as_date(2015-01-20)-
json method GETbody cmd x yheaders
Content-Type applicationjsonX-Custom-Header 1234
varsplayed_at as_date(2015-01-20)
Note the payloads can be defined either in place with the lsquojsonrsquo key or point to an external reference via the lsquofilersquo keyThe file can either reference a local statically served file as shown above or be a URI reference Typically it will be aURI reference to a location in a users source control system where their stubs are stored
The local file references in this example are shown below
stub1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
8 Chapter 3 Stubo Command File
Stub-O-Matic Documentation Release 0X beta
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
request1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
Note that these json payloads for the request and response are defined as strings Stubo also excepts the definition asdictionaries
32 Command Scripting
The YAML file is run through a template processor before being parsed and executed by Stubo Any variables definedsuch as lsquoplayed_atrsquo will evaluated and appropriate subsitutions made
A roll date example
(daterollyaml)
playbackrequests- file dateroll_1433930288_0requestresponse dateroll_1433930288_0stubo_responsevars
getresponse_arg this stub was played at 2015-06-10 095744839438
32 Command Scripting 9
Stub-O-Matic Documentation Release 0X beta
play_date 2014-09-12priority 1putstub_arg this stub was recorded at 2015-06-10 095744839387rec_date 2014-09-10tracking_level full
scenario daterollsession dateroll_1433930288
recordingscenario daterollsession dateroll_1433930288stubs- file dateroll_1433930288_0json
Referenced files
dateroll_1433930288_0json
priority 1args
priority 1rec_date 2014-09-10putstub_arg this stub was recorded at 2015-06-10 095744839387
request
bodyPatterns contains [
ltxml version=10 encoding=UTF-8 standalone=yes gtnltrollmegt n ltOriginDateTimegtroll_date(2014-09-10 as_date(rec_date) as_date(play_date))T000000ZltOriginDateTimegtnltrollmegt]
method POST
response
body ltresponsegtnltputstub_arggt raw putstub_arg ltputstub_arggtnltgetresponse_arggt getresponse_arg ltgetresponse_arggtnltresponsegtstatus 200
dateroll_1433930288_0request
body ltxml version=10 encoding=UTF-8 standalone=yesgtnltrollmegtn ltOriginDateTimegt2014-09-12T000000ZltOriginDateTimegtnltrollmegtheaders host nullpath nullquery uri nullmethod POST
10 Chapter 3 Stubo Command File
CHAPTER 4
Stubo Command File (text format)
The line orientated text version of the command file is an old format It does not support the matching required forREST For example you canrsquot match on the request headers or method It only supports one type of matcher whichis the lsquobody containsrsquo matcher If the text specified for the lsquobody containsrsquo matcher stub is found in the request amatch is found This is often however the only type of matcher required to successfully stub a system This format isstill supported for import via execcmds but will not be exported anymore If a text commands file is imported via alsquocommandsrsquo text file on export via getexport a lsquoyamlrsquo file will be exported
41 Text Command file
bull Lines beginning with a lsquorsquo are treated as comments
bull Commands follow the Stub-O-Matic REST API for example be-ginsessionscenario=abcampsession=12345ampmode=record
bull When the API requires something in the HTTP body input can be sourced by listing files after the command(separated by commas) Files are assumed to be in the same directory as the command file Alternatively a fullURI can be provided to source the input (matcher request or response)
bull Blank lines are ignored
bull An example command is putstubsession=12345rsquofeng_001textMatcher feng_002textMatcherfeng_001response A putstub command must have at least one matcher file and one response file Thisexample has two matchers
What goes in the command files
bull A textMatcher file could be some or all of the text from the request eg AB1234ltFlightInfoQuerygt
bull Multiple textMatchers files may be used in a stub All must evaluate to True against the request for a responseto be returned
bull request files contain the text of a request (eg text xml json) Note the using playback in commands is onlyused to test the stubs as these would be issued by the real system being tested in normal situations
bull response files contain the text of a response (eg text xml json)
bull Any naming convention can be used for matcher request and response files We have used suffixes oftextMatcher request amp response but you are free to use other extensions like matcherxml requestjson re-sponsexml etc
11
Stub-O-Matic Documentation Release 0X beta
42 Command Scripting
Stub-O-Matic supports the ability to load stubs from various and multiple sources This means people can organise allthe stubs that support a particular service or use case together and any test that needs those stubs could pull them infrom common stub libraries maximising stub reuse
set foreign_url = httpwwwgooglecouk set other_url = httpwwwsuncom deletestubsscenario=smartbeginsessionscenario=smartampsession=smart_1ampmode=recordputstubsession=smart_1url=other_urlurl=foreign_urlputstubsession=smart_1text=random_rubbletext=response_textendsessionsession=smart_1
Command files are also programmable with Python code snippets See the example below
set session_name = smart_1 deletestubsscenario=smartbeginsessionscenario=smartampsession=session_nameampmode=recordputstubsession=session_namefirsttextMatcherfirstresponseendsessionsession=session_namebeginsessionscenario=smartampsession=session_nameampmode=playback make 3 requests for i in range(03) getresponsesession=session_namefirstrequest end endsessionsession=session_name
43 Passing Arguments
Arguments can be passed into command files and used in them The following example allows the session and scenarioto be named externally to the command file
execcmdscmdFile=staticcmdstestsacceptsmart_argcommandsampscen=bobampsession=smart_1
The above call can be used as follows
deletestubsscenario=scen[0]beginsessionscenario=scen[0]ampsession= session[0] ampmode=recordputstubsession=session[0]firsttextMatcherfirstresponseendsessionsession=session[0]beginsessionscenario=scen[0]ampsession=session[0]ampmode=playback for i in range(03) getresponsesession=session[0]firstrequest end endsessionsession=session[0]
12 Chapter 4 Stubo Command File (text format)
CHAPTER 5
GUI
51 Tracker
The Stubo lsquoTrackerrsquo page shows details of all requests submitted to Stub-O-Matic and the received response Clickon the link in the function column to show a detailed view Debug level tracking can be added to a getresponse callif lsquotracking_level=fullrsquo is added to the call Matching errors will also trigger some extra debug data to be added to anormal track record
Stubo can not keep tracker data forever so it will eventually drop off the server as space is used up
The tracker page shows all Stubo actions performed on the server Filters are provided to filter on session datetimeresponse times and errors
To go into detail on a particular transaction click the link in the lsquoFunctionrsquo column or lsquoDetailsrsquo in the lsquoStubo Responsersquocolumn For a getresponse call the matchers and responses for the call can be found from the gtgtgt view scenario details(stub matchers and responses) link at the top of the details page
Filtering The tracker filter fields should be self explanatory By default the tracker page shows only data sent to theStubo URL shown at the top of the browser To see tracker items from other URLs on the same hardware tick thelsquoAll Hostsrsquo box
52 Manage
The manage page allows users to execute command files and view system state
The list of loaded stubs includes the date the stubs were loaded into Stubo and the last time the stub was used Clickingthe scenario name shows the stored matchers and responses for the scenario
Also available is a command file scratchpad for entering one or more commands For example one could enter
endsessionsession=first_sessiondeletestubsscenario=firstendsessionsession=second_sessiondeletestubsscenario=second
All the above commands would then be executed See the results on the tracker page
The manage page also shows stubs delay policies and any external modules loaded
13
Stub-O-Matic Documentation Release 0X beta
53 Analytics
The lsquoAnalyticsrsquo tab takes you to a dashboard showing statistics about the Stubo server The orange bar along the topallows you to select different Stubo servers and time periods The first of the 4 squares on the right turns on a legend
Charts
chart title descriptionstubssecondresponse rate
number of stubs requested per second
delay added user defined delays to stubs these can be specified to emulate response times of thesystem being stubbed
latency average time taken to return a stub excluding network and user defined delaysrequestresponse size size in kb of the requests to stubo and responses deliveredhost response latency where Stubo is deployed across multiple servers this chart compares the different servers
against each otherfailure rate percent of failed requests (HTTP response gt= 400)health checks Stubo has a regular health check ping to demonstrate that all systems are working
properlyrunning putstubcount
cumulative total of stubs loaded
running responsecount
cumulative total of stubs retrieved
Also note the Stubo API documentation for getstats
14 Chapter 5 GUI
CHAPTER 6
Integration with Test Tools
Stub-O-Matic works with all major software test execution tools Stubo of course does not run tests or evaluateresults it emulates back-end systems needed by tests
Data dependent software tests typically consist of 3 steps
1 Data setup
2 Test execution
3 Data teardown
Working with Stubo to provide back-end data is no different
Stuborsquos API can be used to setup or teardown data directly Stubo command files offer a shorthand for using the APIand are much more compact Any test tool that can make HTTP calls can use Stubo
61 Test steps example
1 Trigger an http call similar to this from the test tool of your choice
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_setupcommands
2 Run tests
3 Trigger a teardown http call similar to this from the test tool
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_teardowncommands
Setup would typically i) delete stubs ii) set any delays iii) submit a recording via a series of putstub calls and finallyiv) begin a session in playback
Teardown would typically i) end session ii) delete stubs
Setup and teardown can be performed with a mixture of command files (YAML or commands) and direct HTTP APIcalls to stubo
15
Stub-O-Matic Documentation Release 0X beta
16 Chapter 6 Integration with Test Tools
CHAPTER 7
Stubbing
71 Add Delay to a Response
To emulate response times of systems being stubbed one can set a delay on stubs Delays are set on the stub responsenot on the request to stubo This is because one request into the system being tested can result in several back-endstubocalls each of which might need different delay settings
Stubo provides reusable delay_policy objects which can be used across many stubs and across sessions The de-lay_policy can be altered on the fly during a loadperformance test if desired There are currently two delay_policytypes to choose from (1) fixed and (2)random with mean and standard deviation
A fixed delay policy can be created or modified with a commnand such as
putdelay_policyname=a_fixed_delay_policyampdelay_type=fixedampmilliseconds=200
Note that the value for the lsquonamersquo can be set to any string and is used again in the putstub A variable delay can be setup as follows
putdelay_policyname=a_normalvariate_delay_policyampdelay_type=normalvariateampmean=100ampstddev=50
Allowed lsquodelay typersquo values are lsquofixedrsquo and lsquonormalvariatersquo The next step is to use the delay policy when loadingstubs eg
putstubsession=abcampdelay_policy=a_fixed_delay_policy
bull Whenever the stub above is used the named delay policy will be applied
bull Delay policies may be altered on the fly during a test run if desired To change a delay on the fly simply repeatthe putdelay_policy command with a different value Typically this would be done directly by the test toolrather than from a command file
bull Delay policies can be used across multiple sessions This allows for easier alteration of policies where manysessions use the same policy It also means that you should name policies carefully and only alter or sharepolicies you are in control of
72 Matching
Stub-O-Matic currently supports various types of matchers
17
Stub-O-Matic Documentation Release 0X beta
73 Body contains matching
One or more matchers can be defined whereby all matchers must be contained in the request to return a specifiedresponse A typical difficulty with system requests is that they often contain superfluous information such as sessionIDs or time stamps that get in the way of matching a request to the desired response One way Stub-O-Matic can solvethis problem is by the use of lsquomatchersrsquo Matchers are parts of the request that matter to you only what is needed tofind the correct response file
For example a request may include
ltsessionidgt123456ltsessionidgtlttransactionidgt0987654321-9999lttransactionidgt
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Of these 4 lines only the last 2 matter the first 2 lines will differ from one request to the next and be set by the systemmaking the request This request then will be different each time it is made How can one easily match a request likethis to a canned response file without writing code for each requestresponse pair The solution used by Stub-O-Maticis to not store requestresponse file pairs but rather store matchers and the response In the case above the matcherswould be
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Stub-O-Matic contains code which will take a request as input search for the response with the best matchers Everymatcher must be contained in the request to be a match When a match is found the corresponding response is servedback
All matchers must be present in the request to make a match Stub-O-Matic will return the response from the firstmatch if there is more than one possible match Note that whitespace and carriage returns are ignored when matching
Alternatives to removing parts of the request you donrsquot want to match on are
1 using a matcher template to substitute ids or dates etc from the request into the matcher OR
2 using a user exit to run custom code A convenience class XMLManglerExit is provided for XML payloads toignore XMl elements andor attributes from the matching process
74 Templated Matcher
This is an example of a template matcher than extracts values from the request into the matcher The xmltree object isan lxml parsed root element passed into the template
ltrequestgtlttransidgtxmltreexpath(requesttransid)[0]textlttransidgtltdtgt2009-10-24T000000+0000ltdtgtltflightnumbergt0455ltflightnumbergtltrequestgt
75 Templated Responses
A hand crafted Stubo response may contain variables which are substituted in For example a response may contain
set flight_nbr=XX1234 The flight number is flight_nbr
18 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
The resulting response served wil be
The flight number is XX1234
Code may be used for example adding random elements to a response
import random set rnd_nbr = randomrandint(1200) Hello World 1000 + rnd_nbr
The resulting response served will be
Hello World - followed by a number between 1000 and 1200
To put todayrsquos date in a response include the following
today_strformat(dmy)
Stubo responses are run through a Tornado template and any logic or commands allowed in these templates can beused See httpwwwtornadoweborgenstabletemplatehtml for details
76 Request Data in Responses
761 Text Manipulation
It is possible to pull data from the stub request and put it in the response Say you have 200 stubs that are identicalapart from a userid They could be condensed into one stub which uses the userid from the request in the responserequest
ltuseridgtabc123ltuseridgt
matcher
ltuseridgt
response template (using Python)
Hello to request_text[814]
response
Hello to abc123
762 XML Manipulation
Sometimes it is easier to use XPATH to pull data from a request and place it in a response If you need the SessionIdfrom this request
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtJohn1234ltwbsSessionIdgt
Pull the SessionId from the request and use it within the response template
76 Request Data in Responses 19
Stub-O-Matic Documentation Release 0X beta
set namespaces = soapenv httpschemasxmlsoaporgsoapenvelopewbs httpxmlaaacomws200901WBS_Session-20xsd
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtxmltreexpath(soapenvHeaderwbsSessionwbsSessionId
namespaces=namespaces)[0]textltwbsSessionIdgt
Note the xmltree variable is the parsed xml request (as an lxml Root object) made available to the template if therequest is valid xml
Another example - no namespaces in the request Request
ltxml version=10 encoding=UTF-8gtltCompensateCustomersCheck APCWCustom=-1 LocalCurrencyCode=GBP NumberOfAdults=1gt
Response
set currency_code = xmltreexpath(CompensateCustomersCheck)[0]attrib[LocalCurrencyCode] ltresponse LocalCurrencyCode=currency_codegtpay meltresponsegt
77 Stateful Stubs
State is important in simulating back-end systems For example if one was to check-in passenger Bob on a particularflight the back-end would allow this request to succeed only one time and subsequent attempts to check-in the samepassenger on the same flight would be rejected Stubo can emulate this functionality or if desired for repeatable testsStubo can allow Bob to be checked in multiple times on the same flight
Stateful stubbing is enabled automatically If stubs are recorded or loaded in which the same request returns differentresponses Stubo will remember this and internally keep track of which response should come next
If your test is to check-in Bob multiple times simply do not load the stubs representing the check-in rejection andStubo will allow multiple check-ins
78 User Exits
Stub-O-Matic provides hooks into the runtime to execute custom user code Stubo can execute your custom code totransform stub matchers requests andor responses This is a powerful mechanism when your stubs contain dynamicdata or need to change over time This can be useful to perform lsquointelligent stubbingrsquo for example to roll dates Theuser exit API is contained in the stuboextuser_exit module
781 API
782 XML Auto Mangling
If your data is XML a convenience class XMLUserExit is provided to enable auto mangling
20 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
783 API
79 Hooks
710 Modules
Modules are added with the putmodule command and referenced via the ext_module url arg in the putstub call Forexample
putmodulename=staticcmdstestsextxsltmanglerpydeletestubsscenario=mangler_xsltbeginsessionscenario=mangler_xsltampsession=mangler_xslt1ampmode=recordputstubsession=mangler_xslt1ampext_module=mangler firstrequest firstresponseendsessionsession=mangler_xslt1
This example dynamically imports a test mangler from the Stubo server Typically these external python modules thathook into the Stubo runtime would be sourced from source control system using a URL
Note user exit processing is only enabled if the lsquoext_modulersquo variable is set to a pre-loaded module during the putstubcall Each Stub-O-Matic host URL has its own copy of any loaded modules Such modules are available to any testand as such should typically be loaded once per test suite
711 Splitting
Useful to removing random elements from matchers It can happen that the system under test will include seeminglyrandom elements in the request This will cause the next run of the test to not match This problem can be solved byprocessing matchers when loading - during a record A matcher can be split into two with the variable text left outFuture request must match both of the matchers to get the response An example is
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N11111234567806P1ltCommandgtltCryptic_GetScreen_Querygt
Which becomes two matchers
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N1111
and
06P1ltCommandgtltCryptic_GetScreen_Querygt
cutting out the random number lsquo12345678rsquo in the middle
This matcher splitting can be done manually or automatically with a user exit Seestaticcmdstestsextsplitters1firstcommands for and example It uses the rules module at staticcmdstestsextaaapy
Multiple matchers excluding random request elements can be created manually and loaded as such
putstubsession=splitter_record splitter_1textMatcher splitter_2textMatcher splitterresponse
712 Caching Values
If emulating back-end behaviour means writing some code for a particularly tricky behaviour Stubo exposes a simplekey-value cache API to matchers and responses
79 Hooks 21
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
8 Sharing Stubs 23
9 Communicating with Stub-O-Matic 2791 GetResponse URLs 2792 Virtual Stub-O-Matic 27
10 29
11 31
12 33
13 35
14 37
15 39
16 41
17 43
18 45
19 47
20 49
21 Stub-O-Matic API v1 51211 execcmds 51212 getversion 52213 getstatus 53214 beginsession 54215 endsession 54216 endsessions 55217 putscenarios 55218 putstub 55219 getstublist 562110 putdelay_policy 572111 getdelay_policy 582112 deletedelay_policy 582113 getresponse 582114 deletestubs 592115 getexport 592116 getstubcount 612117 putmodule 612118 getmodulelist 622119 deletemodule 622120 deletemodules 622121 Set Tracking Level 632122 Blacklist a host URL 642123 Create Bookmark 642124 Jump to Bookmark 652125 Delete Bookmark 652126 List Bookmarks 652127 getstats 65
ii
22 Examples 67221 Templates Date Rolling 67222 Stateful stubs 67223 Auto Mangling with user exits 67224 user exit using XSLT 68225 User exit using a cache 68
23 Clients (Stubo Integrator) 69231 Test data Setup and Teardown 69232 To Stub or not to Stub 70233 Playback of Stubs 71234 Record Stubs 71235 Performance Requirements 72236 Error Responses 72
24 Reference 73241 Stub Definition 73
25 About 75
26 Change History 77261 stubo Change History 77
27 Indices and tables 79
iii
iv
CHAPTER 1
Introduction
The target of the Stub-O-matic software is to enable automated testing by mastering system dependencies
11 Background
Stub-O-Matic (sometimes called Stubo) is a service which provides data stubs for software testing Stubs enable thesoftware being tested to be isolated from the rest of the environment and 3rd party back-end systems enabling repeatedautomated testing Stubo is capable of supporting functional load and performance testing
Stubbing enables development to progress when the actual dependencies are not available
Stub-O-Matic stubs include just enough intelligence to emulate back-end systems without the complexity often asso-ciated with data set-up activities
The resilience of testing and automated testing is improved with stubs because it is possible to manage stubs moreeasily than 3rd party systems and databases Dependent systems are often not capable of creating the full range of testconditions necessary to fully test code Refer to httpenwikipediaorgwikiTest_stubs for further information
12 Applicability
Use Stub-O-Matic when you donrsquot have as much control over your associated test data to achieve the level of testingrequired Stub-O-Matic utilises the ubiquitous HTTP protocol to receive stub requests and return stubbed responses tothe system under test To use stubs the system under test must be capable of diverting a request that would normallygo to back-end systems How this is best done depends on the architecture of the system under test Implementationsto date have used JAVA AOP Python and ActionalProgress Stub Itineraries
Stubo supports unicode It has not been designed for requests or responses containing bit maps or other non-standardcharacters
1
Stub-O-Matic Documentation Release 0X beta
2 Chapter 1 Introduction
CHAPTER 2
Stub Basics
Stubo simulates back-end systems by matching a request to a response RequestResponse definitions are lsquoloadedrsquo intoStubo
Conventions for this documentation
All Stubo commands are to be prefixed with the server (and port) of the Stubo instance you are using For examplestuboapigetstatus would really be something like lsquohttpmystubostuboapigetstatuslsquo
21 Load and Retrieve Stubs
For the impatient point your browser at stubodefaultexeccmdscmdfile=staticcmdsdemofirstyaml The JSONresponse from Stubo is meant for machines to read For a more human friendly version use the lsquobackrsquo button on yourbrowser and navigate to the lsquoTrackerrsquo page There you will see the most recent Stubo commands and their details Thecommands you just ran from firstyaml are
First use of Stub-O-Matic run this from a browser with uri httpltstubo servergtstubodefaultexeccmdscmdfile=staticcmdsdemofirstyaml
Describe your stubs hererecording
scenario firstsession first_1stubs-json
request method GETbodyPatterns [
contains [get my stub]
]
response
status 200body Hello 1+1 World
3
Stub-O-Matic Documentation Release 0X beta
Provide your requests hereplayback
scenario firstsession first_1requests-
json method GETbody timestamp 092345
get my stub
These will be listed on the Tracker page What just happened The command you made executed a Stub-O-Maticcommand file (demofirstyaml) That file loaded a stub into Stuborsquos database then simulated a request returning theexpected response These actions were logged and displayed on the Tracker page
There are a few key concepts that will make stubbing work
bull Scenario - A re-usable set of stubs
bull Session - The instance or use of the stubs from a scenario
bull Matcher - The matcher used to find the correct response for a request
bull Request - A message normally sent to a back-end by the system under test
bull Response - A message normally returned by a back-end system
bull Stub = Matcher(s) and a response
Stubo commands have been designed to be read written and used by either humans or the Stubo code The stub andcommand files are accessed via a URI (http) In practice this means putting them into a source code repositorysuch as Subversion (SVN) which you should be doing anyway The file location in the execcmds command can belocal to Stubo as in the firstyaml example or (more usual) a URL
For example you could run
stubodefaultexeccmdscmdfile=httpsyour-source-code-repostaticcmdsdemofirstyaml
22 Sessions and Scenarios
In order for stubs to be loaded and retrieved in a predictable automated manner by multiple users many Stubo eventsare controlled by a session A session is a unit of work that must be started and ended Sessions have two possiblemodes of operation
bull record - for loading stubs into the Stubo datastore
bull playback - for retrieving stubbed responses during testing
A scenario is a group of stubs that can be used and re-used by multiple sessions
23 Recording Stubs
Stubs can be created by hand or auto-recorded from a real system under test This is done from what we call a lsquoStubointegrator or Stubo client servicersquo A Stubo integrator is typically a custom library that intercepts real system calls andlsquore-playsrsquo them into a Stubo lsquorecordingrsquo via putstub commands Java and Python Stubo client libraries are provided
4 Chapter 2 Stub Basics
Stub-O-Matic Documentation Release 0X beta
to help integrate Stubo into your applications These clients are maintained in github at httpsgithubcomStub-O-Maticpython-client amp httpsgithubcomStub-O-Maticjava-client
If the real system changes its interface a recording can be repeated to capture the changes Of course this requires thatthe test back-end systems have the correct data in them to be used successfully
Stubo provides mechanisms to deal with context sensitive information such as transaction ids dates user names etcwhich makes recorded stubs less fragile
23 Recording Stubs 5
Stub-O-Matic Documentation Release 0X beta
6 Chapter 2 Stub Basics
CHAPTER 3
Stubo Command File
Command files are used to load stub files and add behaviour to them (state dates etc)
Stubs can also be loaded and tested using individual calls to the Stub-O-Matic server using the Stubo HTTP APICommand files are a shorthand for the API making it simpler to enable repeatable automated interactions with Stub-O-Matic by grouping a set of commands together required for a particular test
31 Command file (YAML)
An example
Stubo YAML Execution ordering is 1) commands 2) recording 3) playback
Commands go herecommands
-putmodule
varsname staticcmdstestsrestyamlnooppy
-putdelay_policy
varsname slowdelay_type fixedmilliseconds 1000
Describe your stubs hererecording
scenario restsession rest_recordingstubs-file stub1jsonvars
recorded_at as_date(2015-01-10)ext_module noop
-json
request method GETbodyPatterns [
7
Stub-O-Matic Documentation Release 0X beta
jsonpath [cmdx]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all ok 2headers Content-Type applicationjsonX-Custom-Header 1234
vars
foo bar
Provide your requests hereplayback
scenario restsession rest_playbackrequests-
file request1jsonvars
played_at as_date(2015-01-20)-
json method GETbody cmd x yheaders
Content-Type applicationjsonX-Custom-Header 1234
varsplayed_at as_date(2015-01-20)
Note the payloads can be defined either in place with the lsquojsonrsquo key or point to an external reference via the lsquofilersquo keyThe file can either reference a local statically served file as shown above or be a URI reference Typically it will be aURI reference to a location in a users source control system where their stubs are stored
The local file references in this example are shown below
stub1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
8 Chapter 3 Stubo Command File
Stub-O-Matic Documentation Release 0X beta
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
request1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
Note that these json payloads for the request and response are defined as strings Stubo also excepts the definition asdictionaries
32 Command Scripting
The YAML file is run through a template processor before being parsed and executed by Stubo Any variables definedsuch as lsquoplayed_atrsquo will evaluated and appropriate subsitutions made
A roll date example
(daterollyaml)
playbackrequests- file dateroll_1433930288_0requestresponse dateroll_1433930288_0stubo_responsevars
getresponse_arg this stub was played at 2015-06-10 095744839438
32 Command Scripting 9
Stub-O-Matic Documentation Release 0X beta
play_date 2014-09-12priority 1putstub_arg this stub was recorded at 2015-06-10 095744839387rec_date 2014-09-10tracking_level full
scenario daterollsession dateroll_1433930288
recordingscenario daterollsession dateroll_1433930288stubs- file dateroll_1433930288_0json
Referenced files
dateroll_1433930288_0json
priority 1args
priority 1rec_date 2014-09-10putstub_arg this stub was recorded at 2015-06-10 095744839387
request
bodyPatterns contains [
ltxml version=10 encoding=UTF-8 standalone=yes gtnltrollmegt n ltOriginDateTimegtroll_date(2014-09-10 as_date(rec_date) as_date(play_date))T000000ZltOriginDateTimegtnltrollmegt]
method POST
response
body ltresponsegtnltputstub_arggt raw putstub_arg ltputstub_arggtnltgetresponse_arggt getresponse_arg ltgetresponse_arggtnltresponsegtstatus 200
dateroll_1433930288_0request
body ltxml version=10 encoding=UTF-8 standalone=yesgtnltrollmegtn ltOriginDateTimegt2014-09-12T000000ZltOriginDateTimegtnltrollmegtheaders host nullpath nullquery uri nullmethod POST
10 Chapter 3 Stubo Command File
CHAPTER 4
Stubo Command File (text format)
The line orientated text version of the command file is an old format It does not support the matching required forREST For example you canrsquot match on the request headers or method It only supports one type of matcher whichis the lsquobody containsrsquo matcher If the text specified for the lsquobody containsrsquo matcher stub is found in the request amatch is found This is often however the only type of matcher required to successfully stub a system This format isstill supported for import via execcmds but will not be exported anymore If a text commands file is imported via alsquocommandsrsquo text file on export via getexport a lsquoyamlrsquo file will be exported
41 Text Command file
bull Lines beginning with a lsquorsquo are treated as comments
bull Commands follow the Stub-O-Matic REST API for example be-ginsessionscenario=abcampsession=12345ampmode=record
bull When the API requires something in the HTTP body input can be sourced by listing files after the command(separated by commas) Files are assumed to be in the same directory as the command file Alternatively a fullURI can be provided to source the input (matcher request or response)
bull Blank lines are ignored
bull An example command is putstubsession=12345rsquofeng_001textMatcher feng_002textMatcherfeng_001response A putstub command must have at least one matcher file and one response file Thisexample has two matchers
What goes in the command files
bull A textMatcher file could be some or all of the text from the request eg AB1234ltFlightInfoQuerygt
bull Multiple textMatchers files may be used in a stub All must evaluate to True against the request for a responseto be returned
bull request files contain the text of a request (eg text xml json) Note the using playback in commands is onlyused to test the stubs as these would be issued by the real system being tested in normal situations
bull response files contain the text of a response (eg text xml json)
bull Any naming convention can be used for matcher request and response files We have used suffixes oftextMatcher request amp response but you are free to use other extensions like matcherxml requestjson re-sponsexml etc
11
Stub-O-Matic Documentation Release 0X beta
42 Command Scripting
Stub-O-Matic supports the ability to load stubs from various and multiple sources This means people can organise allthe stubs that support a particular service or use case together and any test that needs those stubs could pull them infrom common stub libraries maximising stub reuse
set foreign_url = httpwwwgooglecouk set other_url = httpwwwsuncom deletestubsscenario=smartbeginsessionscenario=smartampsession=smart_1ampmode=recordputstubsession=smart_1url=other_urlurl=foreign_urlputstubsession=smart_1text=random_rubbletext=response_textendsessionsession=smart_1
Command files are also programmable with Python code snippets See the example below
set session_name = smart_1 deletestubsscenario=smartbeginsessionscenario=smartampsession=session_nameampmode=recordputstubsession=session_namefirsttextMatcherfirstresponseendsessionsession=session_namebeginsessionscenario=smartampsession=session_nameampmode=playback make 3 requests for i in range(03) getresponsesession=session_namefirstrequest end endsessionsession=session_name
43 Passing Arguments
Arguments can be passed into command files and used in them The following example allows the session and scenarioto be named externally to the command file
execcmdscmdFile=staticcmdstestsacceptsmart_argcommandsampscen=bobampsession=smart_1
The above call can be used as follows
deletestubsscenario=scen[0]beginsessionscenario=scen[0]ampsession= session[0] ampmode=recordputstubsession=session[0]firsttextMatcherfirstresponseendsessionsession=session[0]beginsessionscenario=scen[0]ampsession=session[0]ampmode=playback for i in range(03) getresponsesession=session[0]firstrequest end endsessionsession=session[0]
12 Chapter 4 Stubo Command File (text format)
CHAPTER 5
GUI
51 Tracker
The Stubo lsquoTrackerrsquo page shows details of all requests submitted to Stub-O-Matic and the received response Clickon the link in the function column to show a detailed view Debug level tracking can be added to a getresponse callif lsquotracking_level=fullrsquo is added to the call Matching errors will also trigger some extra debug data to be added to anormal track record
Stubo can not keep tracker data forever so it will eventually drop off the server as space is used up
The tracker page shows all Stubo actions performed on the server Filters are provided to filter on session datetimeresponse times and errors
To go into detail on a particular transaction click the link in the lsquoFunctionrsquo column or lsquoDetailsrsquo in the lsquoStubo Responsersquocolumn For a getresponse call the matchers and responses for the call can be found from the gtgtgt view scenario details(stub matchers and responses) link at the top of the details page
Filtering The tracker filter fields should be self explanatory By default the tracker page shows only data sent to theStubo URL shown at the top of the browser To see tracker items from other URLs on the same hardware tick thelsquoAll Hostsrsquo box
52 Manage
The manage page allows users to execute command files and view system state
The list of loaded stubs includes the date the stubs were loaded into Stubo and the last time the stub was used Clickingthe scenario name shows the stored matchers and responses for the scenario
Also available is a command file scratchpad for entering one or more commands For example one could enter
endsessionsession=first_sessiondeletestubsscenario=firstendsessionsession=second_sessiondeletestubsscenario=second
All the above commands would then be executed See the results on the tracker page
The manage page also shows stubs delay policies and any external modules loaded
13
Stub-O-Matic Documentation Release 0X beta
53 Analytics
The lsquoAnalyticsrsquo tab takes you to a dashboard showing statistics about the Stubo server The orange bar along the topallows you to select different Stubo servers and time periods The first of the 4 squares on the right turns on a legend
Charts
chart title descriptionstubssecondresponse rate
number of stubs requested per second
delay added user defined delays to stubs these can be specified to emulate response times of thesystem being stubbed
latency average time taken to return a stub excluding network and user defined delaysrequestresponse size size in kb of the requests to stubo and responses deliveredhost response latency where Stubo is deployed across multiple servers this chart compares the different servers
against each otherfailure rate percent of failed requests (HTTP response gt= 400)health checks Stubo has a regular health check ping to demonstrate that all systems are working
properlyrunning putstubcount
cumulative total of stubs loaded
running responsecount
cumulative total of stubs retrieved
Also note the Stubo API documentation for getstats
14 Chapter 5 GUI
CHAPTER 6
Integration with Test Tools
Stub-O-Matic works with all major software test execution tools Stubo of course does not run tests or evaluateresults it emulates back-end systems needed by tests
Data dependent software tests typically consist of 3 steps
1 Data setup
2 Test execution
3 Data teardown
Working with Stubo to provide back-end data is no different
Stuborsquos API can be used to setup or teardown data directly Stubo command files offer a shorthand for using the APIand are much more compact Any test tool that can make HTTP calls can use Stubo
61 Test steps example
1 Trigger an http call similar to this from the test tool of your choice
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_setupcommands
2 Run tests
3 Trigger a teardown http call similar to this from the test tool
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_teardowncommands
Setup would typically i) delete stubs ii) set any delays iii) submit a recording via a series of putstub calls and finallyiv) begin a session in playback
Teardown would typically i) end session ii) delete stubs
Setup and teardown can be performed with a mixture of command files (YAML or commands) and direct HTTP APIcalls to stubo
15
Stub-O-Matic Documentation Release 0X beta
16 Chapter 6 Integration with Test Tools
CHAPTER 7
Stubbing
71 Add Delay to a Response
To emulate response times of systems being stubbed one can set a delay on stubs Delays are set on the stub responsenot on the request to stubo This is because one request into the system being tested can result in several back-endstubocalls each of which might need different delay settings
Stubo provides reusable delay_policy objects which can be used across many stubs and across sessions The de-lay_policy can be altered on the fly during a loadperformance test if desired There are currently two delay_policytypes to choose from (1) fixed and (2)random with mean and standard deviation
A fixed delay policy can be created or modified with a commnand such as
putdelay_policyname=a_fixed_delay_policyampdelay_type=fixedampmilliseconds=200
Note that the value for the lsquonamersquo can be set to any string and is used again in the putstub A variable delay can be setup as follows
putdelay_policyname=a_normalvariate_delay_policyampdelay_type=normalvariateampmean=100ampstddev=50
Allowed lsquodelay typersquo values are lsquofixedrsquo and lsquonormalvariatersquo The next step is to use the delay policy when loadingstubs eg
putstubsession=abcampdelay_policy=a_fixed_delay_policy
bull Whenever the stub above is used the named delay policy will be applied
bull Delay policies may be altered on the fly during a test run if desired To change a delay on the fly simply repeatthe putdelay_policy command with a different value Typically this would be done directly by the test toolrather than from a command file
bull Delay policies can be used across multiple sessions This allows for easier alteration of policies where manysessions use the same policy It also means that you should name policies carefully and only alter or sharepolicies you are in control of
72 Matching
Stub-O-Matic currently supports various types of matchers
17
Stub-O-Matic Documentation Release 0X beta
73 Body contains matching
One or more matchers can be defined whereby all matchers must be contained in the request to return a specifiedresponse A typical difficulty with system requests is that they often contain superfluous information such as sessionIDs or time stamps that get in the way of matching a request to the desired response One way Stub-O-Matic can solvethis problem is by the use of lsquomatchersrsquo Matchers are parts of the request that matter to you only what is needed tofind the correct response file
For example a request may include
ltsessionidgt123456ltsessionidgtlttransactionidgt0987654321-9999lttransactionidgt
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Of these 4 lines only the last 2 matter the first 2 lines will differ from one request to the next and be set by the systemmaking the request This request then will be different each time it is made How can one easily match a request likethis to a canned response file without writing code for each requestresponse pair The solution used by Stub-O-Maticis to not store requestresponse file pairs but rather store matchers and the response In the case above the matcherswould be
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Stub-O-Matic contains code which will take a request as input search for the response with the best matchers Everymatcher must be contained in the request to be a match When a match is found the corresponding response is servedback
All matchers must be present in the request to make a match Stub-O-Matic will return the response from the firstmatch if there is more than one possible match Note that whitespace and carriage returns are ignored when matching
Alternatives to removing parts of the request you donrsquot want to match on are
1 using a matcher template to substitute ids or dates etc from the request into the matcher OR
2 using a user exit to run custom code A convenience class XMLManglerExit is provided for XML payloads toignore XMl elements andor attributes from the matching process
74 Templated Matcher
This is an example of a template matcher than extracts values from the request into the matcher The xmltree object isan lxml parsed root element passed into the template
ltrequestgtlttransidgtxmltreexpath(requesttransid)[0]textlttransidgtltdtgt2009-10-24T000000+0000ltdtgtltflightnumbergt0455ltflightnumbergtltrequestgt
75 Templated Responses
A hand crafted Stubo response may contain variables which are substituted in For example a response may contain
set flight_nbr=XX1234 The flight number is flight_nbr
18 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
The resulting response served wil be
The flight number is XX1234
Code may be used for example adding random elements to a response
import random set rnd_nbr = randomrandint(1200) Hello World 1000 + rnd_nbr
The resulting response served will be
Hello World - followed by a number between 1000 and 1200
To put todayrsquos date in a response include the following
today_strformat(dmy)
Stubo responses are run through a Tornado template and any logic or commands allowed in these templates can beused See httpwwwtornadoweborgenstabletemplatehtml for details
76 Request Data in Responses
761 Text Manipulation
It is possible to pull data from the stub request and put it in the response Say you have 200 stubs that are identicalapart from a userid They could be condensed into one stub which uses the userid from the request in the responserequest
ltuseridgtabc123ltuseridgt
matcher
ltuseridgt
response template (using Python)
Hello to request_text[814]
response
Hello to abc123
762 XML Manipulation
Sometimes it is easier to use XPATH to pull data from a request and place it in a response If you need the SessionIdfrom this request
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtJohn1234ltwbsSessionIdgt
Pull the SessionId from the request and use it within the response template
76 Request Data in Responses 19
Stub-O-Matic Documentation Release 0X beta
set namespaces = soapenv httpschemasxmlsoaporgsoapenvelopewbs httpxmlaaacomws200901WBS_Session-20xsd
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtxmltreexpath(soapenvHeaderwbsSessionwbsSessionId
namespaces=namespaces)[0]textltwbsSessionIdgt
Note the xmltree variable is the parsed xml request (as an lxml Root object) made available to the template if therequest is valid xml
Another example - no namespaces in the request Request
ltxml version=10 encoding=UTF-8gtltCompensateCustomersCheck APCWCustom=-1 LocalCurrencyCode=GBP NumberOfAdults=1gt
Response
set currency_code = xmltreexpath(CompensateCustomersCheck)[0]attrib[LocalCurrencyCode] ltresponse LocalCurrencyCode=currency_codegtpay meltresponsegt
77 Stateful Stubs
State is important in simulating back-end systems For example if one was to check-in passenger Bob on a particularflight the back-end would allow this request to succeed only one time and subsequent attempts to check-in the samepassenger on the same flight would be rejected Stubo can emulate this functionality or if desired for repeatable testsStubo can allow Bob to be checked in multiple times on the same flight
Stateful stubbing is enabled automatically If stubs are recorded or loaded in which the same request returns differentresponses Stubo will remember this and internally keep track of which response should come next
If your test is to check-in Bob multiple times simply do not load the stubs representing the check-in rejection andStubo will allow multiple check-ins
78 User Exits
Stub-O-Matic provides hooks into the runtime to execute custom user code Stubo can execute your custom code totransform stub matchers requests andor responses This is a powerful mechanism when your stubs contain dynamicdata or need to change over time This can be useful to perform lsquointelligent stubbingrsquo for example to roll dates Theuser exit API is contained in the stuboextuser_exit module
781 API
782 XML Auto Mangling
If your data is XML a convenience class XMLUserExit is provided to enable auto mangling
20 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
783 API
79 Hooks
710 Modules
Modules are added with the putmodule command and referenced via the ext_module url arg in the putstub call Forexample
putmodulename=staticcmdstestsextxsltmanglerpydeletestubsscenario=mangler_xsltbeginsessionscenario=mangler_xsltampsession=mangler_xslt1ampmode=recordputstubsession=mangler_xslt1ampext_module=mangler firstrequest firstresponseendsessionsession=mangler_xslt1
This example dynamically imports a test mangler from the Stubo server Typically these external python modules thathook into the Stubo runtime would be sourced from source control system using a URL
Note user exit processing is only enabled if the lsquoext_modulersquo variable is set to a pre-loaded module during the putstubcall Each Stub-O-Matic host URL has its own copy of any loaded modules Such modules are available to any testand as such should typically be loaded once per test suite
711 Splitting
Useful to removing random elements from matchers It can happen that the system under test will include seeminglyrandom elements in the request This will cause the next run of the test to not match This problem can be solved byprocessing matchers when loading - during a record A matcher can be split into two with the variable text left outFuture request must match both of the matchers to get the response An example is
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N11111234567806P1ltCommandgtltCryptic_GetScreen_Querygt
Which becomes two matchers
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N1111
and
06P1ltCommandgtltCryptic_GetScreen_Querygt
cutting out the random number lsquo12345678rsquo in the middle
This matcher splitting can be done manually or automatically with a user exit Seestaticcmdstestsextsplitters1firstcommands for and example It uses the rules module at staticcmdstestsextaaapy
Multiple matchers excluding random request elements can be created manually and loaded as such
putstubsession=splitter_record splitter_1textMatcher splitter_2textMatcher splitterresponse
712 Caching Values
If emulating back-end behaviour means writing some code for a particularly tricky behaviour Stubo exposes a simplekey-value cache API to matchers and responses
79 Hooks 21
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
22 Examples 67221 Templates Date Rolling 67222 Stateful stubs 67223 Auto Mangling with user exits 67224 user exit using XSLT 68225 User exit using a cache 68
23 Clients (Stubo Integrator) 69231 Test data Setup and Teardown 69232 To Stub or not to Stub 70233 Playback of Stubs 71234 Record Stubs 71235 Performance Requirements 72236 Error Responses 72
24 Reference 73241 Stub Definition 73
25 About 75
26 Change History 77261 stubo Change History 77
27 Indices and tables 79
iii
iv
CHAPTER 1
Introduction
The target of the Stub-O-matic software is to enable automated testing by mastering system dependencies
11 Background
Stub-O-Matic (sometimes called Stubo) is a service which provides data stubs for software testing Stubs enable thesoftware being tested to be isolated from the rest of the environment and 3rd party back-end systems enabling repeatedautomated testing Stubo is capable of supporting functional load and performance testing
Stubbing enables development to progress when the actual dependencies are not available
Stub-O-Matic stubs include just enough intelligence to emulate back-end systems without the complexity often asso-ciated with data set-up activities
The resilience of testing and automated testing is improved with stubs because it is possible to manage stubs moreeasily than 3rd party systems and databases Dependent systems are often not capable of creating the full range of testconditions necessary to fully test code Refer to httpenwikipediaorgwikiTest_stubs for further information
12 Applicability
Use Stub-O-Matic when you donrsquot have as much control over your associated test data to achieve the level of testingrequired Stub-O-Matic utilises the ubiquitous HTTP protocol to receive stub requests and return stubbed responses tothe system under test To use stubs the system under test must be capable of diverting a request that would normallygo to back-end systems How this is best done depends on the architecture of the system under test Implementationsto date have used JAVA AOP Python and ActionalProgress Stub Itineraries
Stubo supports unicode It has not been designed for requests or responses containing bit maps or other non-standardcharacters
1
Stub-O-Matic Documentation Release 0X beta
2 Chapter 1 Introduction
CHAPTER 2
Stub Basics
Stubo simulates back-end systems by matching a request to a response RequestResponse definitions are lsquoloadedrsquo intoStubo
Conventions for this documentation
All Stubo commands are to be prefixed with the server (and port) of the Stubo instance you are using For examplestuboapigetstatus would really be something like lsquohttpmystubostuboapigetstatuslsquo
21 Load and Retrieve Stubs
For the impatient point your browser at stubodefaultexeccmdscmdfile=staticcmdsdemofirstyaml The JSONresponse from Stubo is meant for machines to read For a more human friendly version use the lsquobackrsquo button on yourbrowser and navigate to the lsquoTrackerrsquo page There you will see the most recent Stubo commands and their details Thecommands you just ran from firstyaml are
First use of Stub-O-Matic run this from a browser with uri httpltstubo servergtstubodefaultexeccmdscmdfile=staticcmdsdemofirstyaml
Describe your stubs hererecording
scenario firstsession first_1stubs-json
request method GETbodyPatterns [
contains [get my stub]
]
response
status 200body Hello 1+1 World
3
Stub-O-Matic Documentation Release 0X beta
Provide your requests hereplayback
scenario firstsession first_1requests-
json method GETbody timestamp 092345
get my stub
These will be listed on the Tracker page What just happened The command you made executed a Stub-O-Maticcommand file (demofirstyaml) That file loaded a stub into Stuborsquos database then simulated a request returning theexpected response These actions were logged and displayed on the Tracker page
There are a few key concepts that will make stubbing work
bull Scenario - A re-usable set of stubs
bull Session - The instance or use of the stubs from a scenario
bull Matcher - The matcher used to find the correct response for a request
bull Request - A message normally sent to a back-end by the system under test
bull Response - A message normally returned by a back-end system
bull Stub = Matcher(s) and a response
Stubo commands have been designed to be read written and used by either humans or the Stubo code The stub andcommand files are accessed via a URI (http) In practice this means putting them into a source code repositorysuch as Subversion (SVN) which you should be doing anyway The file location in the execcmds command can belocal to Stubo as in the firstyaml example or (more usual) a URL
For example you could run
stubodefaultexeccmdscmdfile=httpsyour-source-code-repostaticcmdsdemofirstyaml
22 Sessions and Scenarios
In order for stubs to be loaded and retrieved in a predictable automated manner by multiple users many Stubo eventsare controlled by a session A session is a unit of work that must be started and ended Sessions have two possiblemodes of operation
bull record - for loading stubs into the Stubo datastore
bull playback - for retrieving stubbed responses during testing
A scenario is a group of stubs that can be used and re-used by multiple sessions
23 Recording Stubs
Stubs can be created by hand or auto-recorded from a real system under test This is done from what we call a lsquoStubointegrator or Stubo client servicersquo A Stubo integrator is typically a custom library that intercepts real system calls andlsquore-playsrsquo them into a Stubo lsquorecordingrsquo via putstub commands Java and Python Stubo client libraries are provided
4 Chapter 2 Stub Basics
Stub-O-Matic Documentation Release 0X beta
to help integrate Stubo into your applications These clients are maintained in github at httpsgithubcomStub-O-Maticpython-client amp httpsgithubcomStub-O-Maticjava-client
If the real system changes its interface a recording can be repeated to capture the changes Of course this requires thatthe test back-end systems have the correct data in them to be used successfully
Stubo provides mechanisms to deal with context sensitive information such as transaction ids dates user names etcwhich makes recorded stubs less fragile
23 Recording Stubs 5
Stub-O-Matic Documentation Release 0X beta
6 Chapter 2 Stub Basics
CHAPTER 3
Stubo Command File
Command files are used to load stub files and add behaviour to them (state dates etc)
Stubs can also be loaded and tested using individual calls to the Stub-O-Matic server using the Stubo HTTP APICommand files are a shorthand for the API making it simpler to enable repeatable automated interactions with Stub-O-Matic by grouping a set of commands together required for a particular test
31 Command file (YAML)
An example
Stubo YAML Execution ordering is 1) commands 2) recording 3) playback
Commands go herecommands
-putmodule
varsname staticcmdstestsrestyamlnooppy
-putdelay_policy
varsname slowdelay_type fixedmilliseconds 1000
Describe your stubs hererecording
scenario restsession rest_recordingstubs-file stub1jsonvars
recorded_at as_date(2015-01-10)ext_module noop
-json
request method GETbodyPatterns [
7
Stub-O-Matic Documentation Release 0X beta
jsonpath [cmdx]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all ok 2headers Content-Type applicationjsonX-Custom-Header 1234
vars
foo bar
Provide your requests hereplayback
scenario restsession rest_playbackrequests-
file request1jsonvars
played_at as_date(2015-01-20)-
json method GETbody cmd x yheaders
Content-Type applicationjsonX-Custom-Header 1234
varsplayed_at as_date(2015-01-20)
Note the payloads can be defined either in place with the lsquojsonrsquo key or point to an external reference via the lsquofilersquo keyThe file can either reference a local statically served file as shown above or be a URI reference Typically it will be aURI reference to a location in a users source control system where their stubs are stored
The local file references in this example are shown below
stub1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
8 Chapter 3 Stubo Command File
Stub-O-Matic Documentation Release 0X beta
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
request1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
Note that these json payloads for the request and response are defined as strings Stubo also excepts the definition asdictionaries
32 Command Scripting
The YAML file is run through a template processor before being parsed and executed by Stubo Any variables definedsuch as lsquoplayed_atrsquo will evaluated and appropriate subsitutions made
A roll date example
(daterollyaml)
playbackrequests- file dateroll_1433930288_0requestresponse dateroll_1433930288_0stubo_responsevars
getresponse_arg this stub was played at 2015-06-10 095744839438
32 Command Scripting 9
Stub-O-Matic Documentation Release 0X beta
play_date 2014-09-12priority 1putstub_arg this stub was recorded at 2015-06-10 095744839387rec_date 2014-09-10tracking_level full
scenario daterollsession dateroll_1433930288
recordingscenario daterollsession dateroll_1433930288stubs- file dateroll_1433930288_0json
Referenced files
dateroll_1433930288_0json
priority 1args
priority 1rec_date 2014-09-10putstub_arg this stub was recorded at 2015-06-10 095744839387
request
bodyPatterns contains [
ltxml version=10 encoding=UTF-8 standalone=yes gtnltrollmegt n ltOriginDateTimegtroll_date(2014-09-10 as_date(rec_date) as_date(play_date))T000000ZltOriginDateTimegtnltrollmegt]
method POST
response
body ltresponsegtnltputstub_arggt raw putstub_arg ltputstub_arggtnltgetresponse_arggt getresponse_arg ltgetresponse_arggtnltresponsegtstatus 200
dateroll_1433930288_0request
body ltxml version=10 encoding=UTF-8 standalone=yesgtnltrollmegtn ltOriginDateTimegt2014-09-12T000000ZltOriginDateTimegtnltrollmegtheaders host nullpath nullquery uri nullmethod POST
10 Chapter 3 Stubo Command File
CHAPTER 4
Stubo Command File (text format)
The line orientated text version of the command file is an old format It does not support the matching required forREST For example you canrsquot match on the request headers or method It only supports one type of matcher whichis the lsquobody containsrsquo matcher If the text specified for the lsquobody containsrsquo matcher stub is found in the request amatch is found This is often however the only type of matcher required to successfully stub a system This format isstill supported for import via execcmds but will not be exported anymore If a text commands file is imported via alsquocommandsrsquo text file on export via getexport a lsquoyamlrsquo file will be exported
41 Text Command file
bull Lines beginning with a lsquorsquo are treated as comments
bull Commands follow the Stub-O-Matic REST API for example be-ginsessionscenario=abcampsession=12345ampmode=record
bull When the API requires something in the HTTP body input can be sourced by listing files after the command(separated by commas) Files are assumed to be in the same directory as the command file Alternatively a fullURI can be provided to source the input (matcher request or response)
bull Blank lines are ignored
bull An example command is putstubsession=12345rsquofeng_001textMatcher feng_002textMatcherfeng_001response A putstub command must have at least one matcher file and one response file Thisexample has two matchers
What goes in the command files
bull A textMatcher file could be some or all of the text from the request eg AB1234ltFlightInfoQuerygt
bull Multiple textMatchers files may be used in a stub All must evaluate to True against the request for a responseto be returned
bull request files contain the text of a request (eg text xml json) Note the using playback in commands is onlyused to test the stubs as these would be issued by the real system being tested in normal situations
bull response files contain the text of a response (eg text xml json)
bull Any naming convention can be used for matcher request and response files We have used suffixes oftextMatcher request amp response but you are free to use other extensions like matcherxml requestjson re-sponsexml etc
11
Stub-O-Matic Documentation Release 0X beta
42 Command Scripting
Stub-O-Matic supports the ability to load stubs from various and multiple sources This means people can organise allthe stubs that support a particular service or use case together and any test that needs those stubs could pull them infrom common stub libraries maximising stub reuse
set foreign_url = httpwwwgooglecouk set other_url = httpwwwsuncom deletestubsscenario=smartbeginsessionscenario=smartampsession=smart_1ampmode=recordputstubsession=smart_1url=other_urlurl=foreign_urlputstubsession=smart_1text=random_rubbletext=response_textendsessionsession=smart_1
Command files are also programmable with Python code snippets See the example below
set session_name = smart_1 deletestubsscenario=smartbeginsessionscenario=smartampsession=session_nameampmode=recordputstubsession=session_namefirsttextMatcherfirstresponseendsessionsession=session_namebeginsessionscenario=smartampsession=session_nameampmode=playback make 3 requests for i in range(03) getresponsesession=session_namefirstrequest end endsessionsession=session_name
43 Passing Arguments
Arguments can be passed into command files and used in them The following example allows the session and scenarioto be named externally to the command file
execcmdscmdFile=staticcmdstestsacceptsmart_argcommandsampscen=bobampsession=smart_1
The above call can be used as follows
deletestubsscenario=scen[0]beginsessionscenario=scen[0]ampsession= session[0] ampmode=recordputstubsession=session[0]firsttextMatcherfirstresponseendsessionsession=session[0]beginsessionscenario=scen[0]ampsession=session[0]ampmode=playback for i in range(03) getresponsesession=session[0]firstrequest end endsessionsession=session[0]
12 Chapter 4 Stubo Command File (text format)
CHAPTER 5
GUI
51 Tracker
The Stubo lsquoTrackerrsquo page shows details of all requests submitted to Stub-O-Matic and the received response Clickon the link in the function column to show a detailed view Debug level tracking can be added to a getresponse callif lsquotracking_level=fullrsquo is added to the call Matching errors will also trigger some extra debug data to be added to anormal track record
Stubo can not keep tracker data forever so it will eventually drop off the server as space is used up
The tracker page shows all Stubo actions performed on the server Filters are provided to filter on session datetimeresponse times and errors
To go into detail on a particular transaction click the link in the lsquoFunctionrsquo column or lsquoDetailsrsquo in the lsquoStubo Responsersquocolumn For a getresponse call the matchers and responses for the call can be found from the gtgtgt view scenario details(stub matchers and responses) link at the top of the details page
Filtering The tracker filter fields should be self explanatory By default the tracker page shows only data sent to theStubo URL shown at the top of the browser To see tracker items from other URLs on the same hardware tick thelsquoAll Hostsrsquo box
52 Manage
The manage page allows users to execute command files and view system state
The list of loaded stubs includes the date the stubs were loaded into Stubo and the last time the stub was used Clickingthe scenario name shows the stored matchers and responses for the scenario
Also available is a command file scratchpad for entering one or more commands For example one could enter
endsessionsession=first_sessiondeletestubsscenario=firstendsessionsession=second_sessiondeletestubsscenario=second
All the above commands would then be executed See the results on the tracker page
The manage page also shows stubs delay policies and any external modules loaded
13
Stub-O-Matic Documentation Release 0X beta
53 Analytics
The lsquoAnalyticsrsquo tab takes you to a dashboard showing statistics about the Stubo server The orange bar along the topallows you to select different Stubo servers and time periods The first of the 4 squares on the right turns on a legend
Charts
chart title descriptionstubssecondresponse rate
number of stubs requested per second
delay added user defined delays to stubs these can be specified to emulate response times of thesystem being stubbed
latency average time taken to return a stub excluding network and user defined delaysrequestresponse size size in kb of the requests to stubo and responses deliveredhost response latency where Stubo is deployed across multiple servers this chart compares the different servers
against each otherfailure rate percent of failed requests (HTTP response gt= 400)health checks Stubo has a regular health check ping to demonstrate that all systems are working
properlyrunning putstubcount
cumulative total of stubs loaded
running responsecount
cumulative total of stubs retrieved
Also note the Stubo API documentation for getstats
14 Chapter 5 GUI
CHAPTER 6
Integration with Test Tools
Stub-O-Matic works with all major software test execution tools Stubo of course does not run tests or evaluateresults it emulates back-end systems needed by tests
Data dependent software tests typically consist of 3 steps
1 Data setup
2 Test execution
3 Data teardown
Working with Stubo to provide back-end data is no different
Stuborsquos API can be used to setup or teardown data directly Stubo command files offer a shorthand for using the APIand are much more compact Any test tool that can make HTTP calls can use Stubo
61 Test steps example
1 Trigger an http call similar to this from the test tool of your choice
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_setupcommands
2 Run tests
3 Trigger a teardown http call similar to this from the test tool
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_teardowncommands
Setup would typically i) delete stubs ii) set any delays iii) submit a recording via a series of putstub calls and finallyiv) begin a session in playback
Teardown would typically i) end session ii) delete stubs
Setup and teardown can be performed with a mixture of command files (YAML or commands) and direct HTTP APIcalls to stubo
15
Stub-O-Matic Documentation Release 0X beta
16 Chapter 6 Integration with Test Tools
CHAPTER 7
Stubbing
71 Add Delay to a Response
To emulate response times of systems being stubbed one can set a delay on stubs Delays are set on the stub responsenot on the request to stubo This is because one request into the system being tested can result in several back-endstubocalls each of which might need different delay settings
Stubo provides reusable delay_policy objects which can be used across many stubs and across sessions The de-lay_policy can be altered on the fly during a loadperformance test if desired There are currently two delay_policytypes to choose from (1) fixed and (2)random with mean and standard deviation
A fixed delay policy can be created or modified with a commnand such as
putdelay_policyname=a_fixed_delay_policyampdelay_type=fixedampmilliseconds=200
Note that the value for the lsquonamersquo can be set to any string and is used again in the putstub A variable delay can be setup as follows
putdelay_policyname=a_normalvariate_delay_policyampdelay_type=normalvariateampmean=100ampstddev=50
Allowed lsquodelay typersquo values are lsquofixedrsquo and lsquonormalvariatersquo The next step is to use the delay policy when loadingstubs eg
putstubsession=abcampdelay_policy=a_fixed_delay_policy
bull Whenever the stub above is used the named delay policy will be applied
bull Delay policies may be altered on the fly during a test run if desired To change a delay on the fly simply repeatthe putdelay_policy command with a different value Typically this would be done directly by the test toolrather than from a command file
bull Delay policies can be used across multiple sessions This allows for easier alteration of policies where manysessions use the same policy It also means that you should name policies carefully and only alter or sharepolicies you are in control of
72 Matching
Stub-O-Matic currently supports various types of matchers
17
Stub-O-Matic Documentation Release 0X beta
73 Body contains matching
One or more matchers can be defined whereby all matchers must be contained in the request to return a specifiedresponse A typical difficulty with system requests is that they often contain superfluous information such as sessionIDs or time stamps that get in the way of matching a request to the desired response One way Stub-O-Matic can solvethis problem is by the use of lsquomatchersrsquo Matchers are parts of the request that matter to you only what is needed tofind the correct response file
For example a request may include
ltsessionidgt123456ltsessionidgtlttransactionidgt0987654321-9999lttransactionidgt
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Of these 4 lines only the last 2 matter the first 2 lines will differ from one request to the next and be set by the systemmaking the request This request then will be different each time it is made How can one easily match a request likethis to a canned response file without writing code for each requestresponse pair The solution used by Stub-O-Maticis to not store requestresponse file pairs but rather store matchers and the response In the case above the matcherswould be
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Stub-O-Matic contains code which will take a request as input search for the response with the best matchers Everymatcher must be contained in the request to be a match When a match is found the corresponding response is servedback
All matchers must be present in the request to make a match Stub-O-Matic will return the response from the firstmatch if there is more than one possible match Note that whitespace and carriage returns are ignored when matching
Alternatives to removing parts of the request you donrsquot want to match on are
1 using a matcher template to substitute ids or dates etc from the request into the matcher OR
2 using a user exit to run custom code A convenience class XMLManglerExit is provided for XML payloads toignore XMl elements andor attributes from the matching process
74 Templated Matcher
This is an example of a template matcher than extracts values from the request into the matcher The xmltree object isan lxml parsed root element passed into the template
ltrequestgtlttransidgtxmltreexpath(requesttransid)[0]textlttransidgtltdtgt2009-10-24T000000+0000ltdtgtltflightnumbergt0455ltflightnumbergtltrequestgt
75 Templated Responses
A hand crafted Stubo response may contain variables which are substituted in For example a response may contain
set flight_nbr=XX1234 The flight number is flight_nbr
18 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
The resulting response served wil be
The flight number is XX1234
Code may be used for example adding random elements to a response
import random set rnd_nbr = randomrandint(1200) Hello World 1000 + rnd_nbr
The resulting response served will be
Hello World - followed by a number between 1000 and 1200
To put todayrsquos date in a response include the following
today_strformat(dmy)
Stubo responses are run through a Tornado template and any logic or commands allowed in these templates can beused See httpwwwtornadoweborgenstabletemplatehtml for details
76 Request Data in Responses
761 Text Manipulation
It is possible to pull data from the stub request and put it in the response Say you have 200 stubs that are identicalapart from a userid They could be condensed into one stub which uses the userid from the request in the responserequest
ltuseridgtabc123ltuseridgt
matcher
ltuseridgt
response template (using Python)
Hello to request_text[814]
response
Hello to abc123
762 XML Manipulation
Sometimes it is easier to use XPATH to pull data from a request and place it in a response If you need the SessionIdfrom this request
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtJohn1234ltwbsSessionIdgt
Pull the SessionId from the request and use it within the response template
76 Request Data in Responses 19
Stub-O-Matic Documentation Release 0X beta
set namespaces = soapenv httpschemasxmlsoaporgsoapenvelopewbs httpxmlaaacomws200901WBS_Session-20xsd
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtxmltreexpath(soapenvHeaderwbsSessionwbsSessionId
namespaces=namespaces)[0]textltwbsSessionIdgt
Note the xmltree variable is the parsed xml request (as an lxml Root object) made available to the template if therequest is valid xml
Another example - no namespaces in the request Request
ltxml version=10 encoding=UTF-8gtltCompensateCustomersCheck APCWCustom=-1 LocalCurrencyCode=GBP NumberOfAdults=1gt
Response
set currency_code = xmltreexpath(CompensateCustomersCheck)[0]attrib[LocalCurrencyCode] ltresponse LocalCurrencyCode=currency_codegtpay meltresponsegt
77 Stateful Stubs
State is important in simulating back-end systems For example if one was to check-in passenger Bob on a particularflight the back-end would allow this request to succeed only one time and subsequent attempts to check-in the samepassenger on the same flight would be rejected Stubo can emulate this functionality or if desired for repeatable testsStubo can allow Bob to be checked in multiple times on the same flight
Stateful stubbing is enabled automatically If stubs are recorded or loaded in which the same request returns differentresponses Stubo will remember this and internally keep track of which response should come next
If your test is to check-in Bob multiple times simply do not load the stubs representing the check-in rejection andStubo will allow multiple check-ins
78 User Exits
Stub-O-Matic provides hooks into the runtime to execute custom user code Stubo can execute your custom code totransform stub matchers requests andor responses This is a powerful mechanism when your stubs contain dynamicdata or need to change over time This can be useful to perform lsquointelligent stubbingrsquo for example to roll dates Theuser exit API is contained in the stuboextuser_exit module
781 API
782 XML Auto Mangling
If your data is XML a convenience class XMLUserExit is provided to enable auto mangling
20 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
783 API
79 Hooks
710 Modules
Modules are added with the putmodule command and referenced via the ext_module url arg in the putstub call Forexample
putmodulename=staticcmdstestsextxsltmanglerpydeletestubsscenario=mangler_xsltbeginsessionscenario=mangler_xsltampsession=mangler_xslt1ampmode=recordputstubsession=mangler_xslt1ampext_module=mangler firstrequest firstresponseendsessionsession=mangler_xslt1
This example dynamically imports a test mangler from the Stubo server Typically these external python modules thathook into the Stubo runtime would be sourced from source control system using a URL
Note user exit processing is only enabled if the lsquoext_modulersquo variable is set to a pre-loaded module during the putstubcall Each Stub-O-Matic host URL has its own copy of any loaded modules Such modules are available to any testand as such should typically be loaded once per test suite
711 Splitting
Useful to removing random elements from matchers It can happen that the system under test will include seeminglyrandom elements in the request This will cause the next run of the test to not match This problem can be solved byprocessing matchers when loading - during a record A matcher can be split into two with the variable text left outFuture request must match both of the matchers to get the response An example is
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N11111234567806P1ltCommandgtltCryptic_GetScreen_Querygt
Which becomes two matchers
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N1111
and
06P1ltCommandgtltCryptic_GetScreen_Querygt
cutting out the random number lsquo12345678rsquo in the middle
This matcher splitting can be done manually or automatically with a user exit Seestaticcmdstestsextsplitters1firstcommands for and example It uses the rules module at staticcmdstestsextaaapy
Multiple matchers excluding random request elements can be created manually and loaded as such
putstubsession=splitter_record splitter_1textMatcher splitter_2textMatcher splitterresponse
712 Caching Values
If emulating back-end behaviour means writing some code for a particularly tricky behaviour Stubo exposes a simplekey-value cache API to matchers and responses
79 Hooks 21
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
iv
CHAPTER 1
Introduction
The target of the Stub-O-matic software is to enable automated testing by mastering system dependencies
11 Background
Stub-O-Matic (sometimes called Stubo) is a service which provides data stubs for software testing Stubs enable thesoftware being tested to be isolated from the rest of the environment and 3rd party back-end systems enabling repeatedautomated testing Stubo is capable of supporting functional load and performance testing
Stubbing enables development to progress when the actual dependencies are not available
Stub-O-Matic stubs include just enough intelligence to emulate back-end systems without the complexity often asso-ciated with data set-up activities
The resilience of testing and automated testing is improved with stubs because it is possible to manage stubs moreeasily than 3rd party systems and databases Dependent systems are often not capable of creating the full range of testconditions necessary to fully test code Refer to httpenwikipediaorgwikiTest_stubs for further information
12 Applicability
Use Stub-O-Matic when you donrsquot have as much control over your associated test data to achieve the level of testingrequired Stub-O-Matic utilises the ubiquitous HTTP protocol to receive stub requests and return stubbed responses tothe system under test To use stubs the system under test must be capable of diverting a request that would normallygo to back-end systems How this is best done depends on the architecture of the system under test Implementationsto date have used JAVA AOP Python and ActionalProgress Stub Itineraries
Stubo supports unicode It has not been designed for requests or responses containing bit maps or other non-standardcharacters
1
Stub-O-Matic Documentation Release 0X beta
2 Chapter 1 Introduction
CHAPTER 2
Stub Basics
Stubo simulates back-end systems by matching a request to a response RequestResponse definitions are lsquoloadedrsquo intoStubo
Conventions for this documentation
All Stubo commands are to be prefixed with the server (and port) of the Stubo instance you are using For examplestuboapigetstatus would really be something like lsquohttpmystubostuboapigetstatuslsquo
21 Load and Retrieve Stubs
For the impatient point your browser at stubodefaultexeccmdscmdfile=staticcmdsdemofirstyaml The JSONresponse from Stubo is meant for machines to read For a more human friendly version use the lsquobackrsquo button on yourbrowser and navigate to the lsquoTrackerrsquo page There you will see the most recent Stubo commands and their details Thecommands you just ran from firstyaml are
First use of Stub-O-Matic run this from a browser with uri httpltstubo servergtstubodefaultexeccmdscmdfile=staticcmdsdemofirstyaml
Describe your stubs hererecording
scenario firstsession first_1stubs-json
request method GETbodyPatterns [
contains [get my stub]
]
response
status 200body Hello 1+1 World
3
Stub-O-Matic Documentation Release 0X beta
Provide your requests hereplayback
scenario firstsession first_1requests-
json method GETbody timestamp 092345
get my stub
These will be listed on the Tracker page What just happened The command you made executed a Stub-O-Maticcommand file (demofirstyaml) That file loaded a stub into Stuborsquos database then simulated a request returning theexpected response These actions were logged and displayed on the Tracker page
There are a few key concepts that will make stubbing work
bull Scenario - A re-usable set of stubs
bull Session - The instance or use of the stubs from a scenario
bull Matcher - The matcher used to find the correct response for a request
bull Request - A message normally sent to a back-end by the system under test
bull Response - A message normally returned by a back-end system
bull Stub = Matcher(s) and a response
Stubo commands have been designed to be read written and used by either humans or the Stubo code The stub andcommand files are accessed via a URI (http) In practice this means putting them into a source code repositorysuch as Subversion (SVN) which you should be doing anyway The file location in the execcmds command can belocal to Stubo as in the firstyaml example or (more usual) a URL
For example you could run
stubodefaultexeccmdscmdfile=httpsyour-source-code-repostaticcmdsdemofirstyaml
22 Sessions and Scenarios
In order for stubs to be loaded and retrieved in a predictable automated manner by multiple users many Stubo eventsare controlled by a session A session is a unit of work that must be started and ended Sessions have two possiblemodes of operation
bull record - for loading stubs into the Stubo datastore
bull playback - for retrieving stubbed responses during testing
A scenario is a group of stubs that can be used and re-used by multiple sessions
23 Recording Stubs
Stubs can be created by hand or auto-recorded from a real system under test This is done from what we call a lsquoStubointegrator or Stubo client servicersquo A Stubo integrator is typically a custom library that intercepts real system calls andlsquore-playsrsquo them into a Stubo lsquorecordingrsquo via putstub commands Java and Python Stubo client libraries are provided
4 Chapter 2 Stub Basics
Stub-O-Matic Documentation Release 0X beta
to help integrate Stubo into your applications These clients are maintained in github at httpsgithubcomStub-O-Maticpython-client amp httpsgithubcomStub-O-Maticjava-client
If the real system changes its interface a recording can be repeated to capture the changes Of course this requires thatthe test back-end systems have the correct data in them to be used successfully
Stubo provides mechanisms to deal with context sensitive information such as transaction ids dates user names etcwhich makes recorded stubs less fragile
23 Recording Stubs 5
Stub-O-Matic Documentation Release 0X beta
6 Chapter 2 Stub Basics
CHAPTER 3
Stubo Command File
Command files are used to load stub files and add behaviour to them (state dates etc)
Stubs can also be loaded and tested using individual calls to the Stub-O-Matic server using the Stubo HTTP APICommand files are a shorthand for the API making it simpler to enable repeatable automated interactions with Stub-O-Matic by grouping a set of commands together required for a particular test
31 Command file (YAML)
An example
Stubo YAML Execution ordering is 1) commands 2) recording 3) playback
Commands go herecommands
-putmodule
varsname staticcmdstestsrestyamlnooppy
-putdelay_policy
varsname slowdelay_type fixedmilliseconds 1000
Describe your stubs hererecording
scenario restsession rest_recordingstubs-file stub1jsonvars
recorded_at as_date(2015-01-10)ext_module noop
-json
request method GETbodyPatterns [
7
Stub-O-Matic Documentation Release 0X beta
jsonpath [cmdx]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all ok 2headers Content-Type applicationjsonX-Custom-Header 1234
vars
foo bar
Provide your requests hereplayback
scenario restsession rest_playbackrequests-
file request1jsonvars
played_at as_date(2015-01-20)-
json method GETbody cmd x yheaders
Content-Type applicationjsonX-Custom-Header 1234
varsplayed_at as_date(2015-01-20)
Note the payloads can be defined either in place with the lsquojsonrsquo key or point to an external reference via the lsquofilersquo keyThe file can either reference a local statically served file as shown above or be a URI reference Typically it will be aURI reference to a location in a users source control system where their stubs are stored
The local file references in this example are shown below
stub1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
8 Chapter 3 Stubo Command File
Stub-O-Matic Documentation Release 0X beta
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
request1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
Note that these json payloads for the request and response are defined as strings Stubo also excepts the definition asdictionaries
32 Command Scripting
The YAML file is run through a template processor before being parsed and executed by Stubo Any variables definedsuch as lsquoplayed_atrsquo will evaluated and appropriate subsitutions made
A roll date example
(daterollyaml)
playbackrequests- file dateroll_1433930288_0requestresponse dateroll_1433930288_0stubo_responsevars
getresponse_arg this stub was played at 2015-06-10 095744839438
32 Command Scripting 9
Stub-O-Matic Documentation Release 0X beta
play_date 2014-09-12priority 1putstub_arg this stub was recorded at 2015-06-10 095744839387rec_date 2014-09-10tracking_level full
scenario daterollsession dateroll_1433930288
recordingscenario daterollsession dateroll_1433930288stubs- file dateroll_1433930288_0json
Referenced files
dateroll_1433930288_0json
priority 1args
priority 1rec_date 2014-09-10putstub_arg this stub was recorded at 2015-06-10 095744839387
request
bodyPatterns contains [
ltxml version=10 encoding=UTF-8 standalone=yes gtnltrollmegt n ltOriginDateTimegtroll_date(2014-09-10 as_date(rec_date) as_date(play_date))T000000ZltOriginDateTimegtnltrollmegt]
method POST
response
body ltresponsegtnltputstub_arggt raw putstub_arg ltputstub_arggtnltgetresponse_arggt getresponse_arg ltgetresponse_arggtnltresponsegtstatus 200
dateroll_1433930288_0request
body ltxml version=10 encoding=UTF-8 standalone=yesgtnltrollmegtn ltOriginDateTimegt2014-09-12T000000ZltOriginDateTimegtnltrollmegtheaders host nullpath nullquery uri nullmethod POST
10 Chapter 3 Stubo Command File
CHAPTER 4
Stubo Command File (text format)
The line orientated text version of the command file is an old format It does not support the matching required forREST For example you canrsquot match on the request headers or method It only supports one type of matcher whichis the lsquobody containsrsquo matcher If the text specified for the lsquobody containsrsquo matcher stub is found in the request amatch is found This is often however the only type of matcher required to successfully stub a system This format isstill supported for import via execcmds but will not be exported anymore If a text commands file is imported via alsquocommandsrsquo text file on export via getexport a lsquoyamlrsquo file will be exported
41 Text Command file
bull Lines beginning with a lsquorsquo are treated as comments
bull Commands follow the Stub-O-Matic REST API for example be-ginsessionscenario=abcampsession=12345ampmode=record
bull When the API requires something in the HTTP body input can be sourced by listing files after the command(separated by commas) Files are assumed to be in the same directory as the command file Alternatively a fullURI can be provided to source the input (matcher request or response)
bull Blank lines are ignored
bull An example command is putstubsession=12345rsquofeng_001textMatcher feng_002textMatcherfeng_001response A putstub command must have at least one matcher file and one response file Thisexample has two matchers
What goes in the command files
bull A textMatcher file could be some or all of the text from the request eg AB1234ltFlightInfoQuerygt
bull Multiple textMatchers files may be used in a stub All must evaluate to True against the request for a responseto be returned
bull request files contain the text of a request (eg text xml json) Note the using playback in commands is onlyused to test the stubs as these would be issued by the real system being tested in normal situations
bull response files contain the text of a response (eg text xml json)
bull Any naming convention can be used for matcher request and response files We have used suffixes oftextMatcher request amp response but you are free to use other extensions like matcherxml requestjson re-sponsexml etc
11
Stub-O-Matic Documentation Release 0X beta
42 Command Scripting
Stub-O-Matic supports the ability to load stubs from various and multiple sources This means people can organise allthe stubs that support a particular service or use case together and any test that needs those stubs could pull them infrom common stub libraries maximising stub reuse
set foreign_url = httpwwwgooglecouk set other_url = httpwwwsuncom deletestubsscenario=smartbeginsessionscenario=smartampsession=smart_1ampmode=recordputstubsession=smart_1url=other_urlurl=foreign_urlputstubsession=smart_1text=random_rubbletext=response_textendsessionsession=smart_1
Command files are also programmable with Python code snippets See the example below
set session_name = smart_1 deletestubsscenario=smartbeginsessionscenario=smartampsession=session_nameampmode=recordputstubsession=session_namefirsttextMatcherfirstresponseendsessionsession=session_namebeginsessionscenario=smartampsession=session_nameampmode=playback make 3 requests for i in range(03) getresponsesession=session_namefirstrequest end endsessionsession=session_name
43 Passing Arguments
Arguments can be passed into command files and used in them The following example allows the session and scenarioto be named externally to the command file
execcmdscmdFile=staticcmdstestsacceptsmart_argcommandsampscen=bobampsession=smart_1
The above call can be used as follows
deletestubsscenario=scen[0]beginsessionscenario=scen[0]ampsession= session[0] ampmode=recordputstubsession=session[0]firsttextMatcherfirstresponseendsessionsession=session[0]beginsessionscenario=scen[0]ampsession=session[0]ampmode=playback for i in range(03) getresponsesession=session[0]firstrequest end endsessionsession=session[0]
12 Chapter 4 Stubo Command File (text format)
CHAPTER 5
GUI
51 Tracker
The Stubo lsquoTrackerrsquo page shows details of all requests submitted to Stub-O-Matic and the received response Clickon the link in the function column to show a detailed view Debug level tracking can be added to a getresponse callif lsquotracking_level=fullrsquo is added to the call Matching errors will also trigger some extra debug data to be added to anormal track record
Stubo can not keep tracker data forever so it will eventually drop off the server as space is used up
The tracker page shows all Stubo actions performed on the server Filters are provided to filter on session datetimeresponse times and errors
To go into detail on a particular transaction click the link in the lsquoFunctionrsquo column or lsquoDetailsrsquo in the lsquoStubo Responsersquocolumn For a getresponse call the matchers and responses for the call can be found from the gtgtgt view scenario details(stub matchers and responses) link at the top of the details page
Filtering The tracker filter fields should be self explanatory By default the tracker page shows only data sent to theStubo URL shown at the top of the browser To see tracker items from other URLs on the same hardware tick thelsquoAll Hostsrsquo box
52 Manage
The manage page allows users to execute command files and view system state
The list of loaded stubs includes the date the stubs were loaded into Stubo and the last time the stub was used Clickingthe scenario name shows the stored matchers and responses for the scenario
Also available is a command file scratchpad for entering one or more commands For example one could enter
endsessionsession=first_sessiondeletestubsscenario=firstendsessionsession=second_sessiondeletestubsscenario=second
All the above commands would then be executed See the results on the tracker page
The manage page also shows stubs delay policies and any external modules loaded
13
Stub-O-Matic Documentation Release 0X beta
53 Analytics
The lsquoAnalyticsrsquo tab takes you to a dashboard showing statistics about the Stubo server The orange bar along the topallows you to select different Stubo servers and time periods The first of the 4 squares on the right turns on a legend
Charts
chart title descriptionstubssecondresponse rate
number of stubs requested per second
delay added user defined delays to stubs these can be specified to emulate response times of thesystem being stubbed
latency average time taken to return a stub excluding network and user defined delaysrequestresponse size size in kb of the requests to stubo and responses deliveredhost response latency where Stubo is deployed across multiple servers this chart compares the different servers
against each otherfailure rate percent of failed requests (HTTP response gt= 400)health checks Stubo has a regular health check ping to demonstrate that all systems are working
properlyrunning putstubcount
cumulative total of stubs loaded
running responsecount
cumulative total of stubs retrieved
Also note the Stubo API documentation for getstats
14 Chapter 5 GUI
CHAPTER 6
Integration with Test Tools
Stub-O-Matic works with all major software test execution tools Stubo of course does not run tests or evaluateresults it emulates back-end systems needed by tests
Data dependent software tests typically consist of 3 steps
1 Data setup
2 Test execution
3 Data teardown
Working with Stubo to provide back-end data is no different
Stuborsquos API can be used to setup or teardown data directly Stubo command files offer a shorthand for using the APIand are much more compact Any test tool that can make HTTP calls can use Stubo
61 Test steps example
1 Trigger an http call similar to this from the test tool of your choice
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_setupcommands
2 Run tests
3 Trigger a teardown http call similar to this from the test tool
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_teardowncommands
Setup would typically i) delete stubs ii) set any delays iii) submit a recording via a series of putstub calls and finallyiv) begin a session in playback
Teardown would typically i) end session ii) delete stubs
Setup and teardown can be performed with a mixture of command files (YAML or commands) and direct HTTP APIcalls to stubo
15
Stub-O-Matic Documentation Release 0X beta
16 Chapter 6 Integration with Test Tools
CHAPTER 7
Stubbing
71 Add Delay to a Response
To emulate response times of systems being stubbed one can set a delay on stubs Delays are set on the stub responsenot on the request to stubo This is because one request into the system being tested can result in several back-endstubocalls each of which might need different delay settings
Stubo provides reusable delay_policy objects which can be used across many stubs and across sessions The de-lay_policy can be altered on the fly during a loadperformance test if desired There are currently two delay_policytypes to choose from (1) fixed and (2)random with mean and standard deviation
A fixed delay policy can be created or modified with a commnand such as
putdelay_policyname=a_fixed_delay_policyampdelay_type=fixedampmilliseconds=200
Note that the value for the lsquonamersquo can be set to any string and is used again in the putstub A variable delay can be setup as follows
putdelay_policyname=a_normalvariate_delay_policyampdelay_type=normalvariateampmean=100ampstddev=50
Allowed lsquodelay typersquo values are lsquofixedrsquo and lsquonormalvariatersquo The next step is to use the delay policy when loadingstubs eg
putstubsession=abcampdelay_policy=a_fixed_delay_policy
bull Whenever the stub above is used the named delay policy will be applied
bull Delay policies may be altered on the fly during a test run if desired To change a delay on the fly simply repeatthe putdelay_policy command with a different value Typically this would be done directly by the test toolrather than from a command file
bull Delay policies can be used across multiple sessions This allows for easier alteration of policies where manysessions use the same policy It also means that you should name policies carefully and only alter or sharepolicies you are in control of
72 Matching
Stub-O-Matic currently supports various types of matchers
17
Stub-O-Matic Documentation Release 0X beta
73 Body contains matching
One or more matchers can be defined whereby all matchers must be contained in the request to return a specifiedresponse A typical difficulty with system requests is that they often contain superfluous information such as sessionIDs or time stamps that get in the way of matching a request to the desired response One way Stub-O-Matic can solvethis problem is by the use of lsquomatchersrsquo Matchers are parts of the request that matter to you only what is needed tofind the correct response file
For example a request may include
ltsessionidgt123456ltsessionidgtlttransactionidgt0987654321-9999lttransactionidgt
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Of these 4 lines only the last 2 matter the first 2 lines will differ from one request to the next and be set by the systemmaking the request This request then will be different each time it is made How can one easily match a request likethis to a canned response file without writing code for each requestresponse pair The solution used by Stub-O-Maticis to not store requestresponse file pairs but rather store matchers and the response In the case above the matcherswould be
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Stub-O-Matic contains code which will take a request as input search for the response with the best matchers Everymatcher must be contained in the request to be a match When a match is found the corresponding response is servedback
All matchers must be present in the request to make a match Stub-O-Matic will return the response from the firstmatch if there is more than one possible match Note that whitespace and carriage returns are ignored when matching
Alternatives to removing parts of the request you donrsquot want to match on are
1 using a matcher template to substitute ids or dates etc from the request into the matcher OR
2 using a user exit to run custom code A convenience class XMLManglerExit is provided for XML payloads toignore XMl elements andor attributes from the matching process
74 Templated Matcher
This is an example of a template matcher than extracts values from the request into the matcher The xmltree object isan lxml parsed root element passed into the template
ltrequestgtlttransidgtxmltreexpath(requesttransid)[0]textlttransidgtltdtgt2009-10-24T000000+0000ltdtgtltflightnumbergt0455ltflightnumbergtltrequestgt
75 Templated Responses
A hand crafted Stubo response may contain variables which are substituted in For example a response may contain
set flight_nbr=XX1234 The flight number is flight_nbr
18 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
The resulting response served wil be
The flight number is XX1234
Code may be used for example adding random elements to a response
import random set rnd_nbr = randomrandint(1200) Hello World 1000 + rnd_nbr
The resulting response served will be
Hello World - followed by a number between 1000 and 1200
To put todayrsquos date in a response include the following
today_strformat(dmy)
Stubo responses are run through a Tornado template and any logic or commands allowed in these templates can beused See httpwwwtornadoweborgenstabletemplatehtml for details
76 Request Data in Responses
761 Text Manipulation
It is possible to pull data from the stub request and put it in the response Say you have 200 stubs that are identicalapart from a userid They could be condensed into one stub which uses the userid from the request in the responserequest
ltuseridgtabc123ltuseridgt
matcher
ltuseridgt
response template (using Python)
Hello to request_text[814]
response
Hello to abc123
762 XML Manipulation
Sometimes it is easier to use XPATH to pull data from a request and place it in a response If you need the SessionIdfrom this request
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtJohn1234ltwbsSessionIdgt
Pull the SessionId from the request and use it within the response template
76 Request Data in Responses 19
Stub-O-Matic Documentation Release 0X beta
set namespaces = soapenv httpschemasxmlsoaporgsoapenvelopewbs httpxmlaaacomws200901WBS_Session-20xsd
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtxmltreexpath(soapenvHeaderwbsSessionwbsSessionId
namespaces=namespaces)[0]textltwbsSessionIdgt
Note the xmltree variable is the parsed xml request (as an lxml Root object) made available to the template if therequest is valid xml
Another example - no namespaces in the request Request
ltxml version=10 encoding=UTF-8gtltCompensateCustomersCheck APCWCustom=-1 LocalCurrencyCode=GBP NumberOfAdults=1gt
Response
set currency_code = xmltreexpath(CompensateCustomersCheck)[0]attrib[LocalCurrencyCode] ltresponse LocalCurrencyCode=currency_codegtpay meltresponsegt
77 Stateful Stubs
State is important in simulating back-end systems For example if one was to check-in passenger Bob on a particularflight the back-end would allow this request to succeed only one time and subsequent attempts to check-in the samepassenger on the same flight would be rejected Stubo can emulate this functionality or if desired for repeatable testsStubo can allow Bob to be checked in multiple times on the same flight
Stateful stubbing is enabled automatically If stubs are recorded or loaded in which the same request returns differentresponses Stubo will remember this and internally keep track of which response should come next
If your test is to check-in Bob multiple times simply do not load the stubs representing the check-in rejection andStubo will allow multiple check-ins
78 User Exits
Stub-O-Matic provides hooks into the runtime to execute custom user code Stubo can execute your custom code totransform stub matchers requests andor responses This is a powerful mechanism when your stubs contain dynamicdata or need to change over time This can be useful to perform lsquointelligent stubbingrsquo for example to roll dates Theuser exit API is contained in the stuboextuser_exit module
781 API
782 XML Auto Mangling
If your data is XML a convenience class XMLUserExit is provided to enable auto mangling
20 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
783 API
79 Hooks
710 Modules
Modules are added with the putmodule command and referenced via the ext_module url arg in the putstub call Forexample
putmodulename=staticcmdstestsextxsltmanglerpydeletestubsscenario=mangler_xsltbeginsessionscenario=mangler_xsltampsession=mangler_xslt1ampmode=recordputstubsession=mangler_xslt1ampext_module=mangler firstrequest firstresponseendsessionsession=mangler_xslt1
This example dynamically imports a test mangler from the Stubo server Typically these external python modules thathook into the Stubo runtime would be sourced from source control system using a URL
Note user exit processing is only enabled if the lsquoext_modulersquo variable is set to a pre-loaded module during the putstubcall Each Stub-O-Matic host URL has its own copy of any loaded modules Such modules are available to any testand as such should typically be loaded once per test suite
711 Splitting
Useful to removing random elements from matchers It can happen that the system under test will include seeminglyrandom elements in the request This will cause the next run of the test to not match This problem can be solved byprocessing matchers when loading - during a record A matcher can be split into two with the variable text left outFuture request must match both of the matchers to get the response An example is
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N11111234567806P1ltCommandgtltCryptic_GetScreen_Querygt
Which becomes two matchers
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N1111
and
06P1ltCommandgtltCryptic_GetScreen_Querygt
cutting out the random number lsquo12345678rsquo in the middle
This matcher splitting can be done manually or automatically with a user exit Seestaticcmdstestsextsplitters1firstcommands for and example It uses the rules module at staticcmdstestsextaaapy
Multiple matchers excluding random request elements can be created manually and loaded as such
putstubsession=splitter_record splitter_1textMatcher splitter_2textMatcher splitterresponse
712 Caching Values
If emulating back-end behaviour means writing some code for a particularly tricky behaviour Stubo exposes a simplekey-value cache API to matchers and responses
79 Hooks 21
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
CHAPTER 1
Introduction
The target of the Stub-O-matic software is to enable automated testing by mastering system dependencies
11 Background
Stub-O-Matic (sometimes called Stubo) is a service which provides data stubs for software testing Stubs enable thesoftware being tested to be isolated from the rest of the environment and 3rd party back-end systems enabling repeatedautomated testing Stubo is capable of supporting functional load and performance testing
Stubbing enables development to progress when the actual dependencies are not available
Stub-O-Matic stubs include just enough intelligence to emulate back-end systems without the complexity often asso-ciated with data set-up activities
The resilience of testing and automated testing is improved with stubs because it is possible to manage stubs moreeasily than 3rd party systems and databases Dependent systems are often not capable of creating the full range of testconditions necessary to fully test code Refer to httpenwikipediaorgwikiTest_stubs for further information
12 Applicability
Use Stub-O-Matic when you donrsquot have as much control over your associated test data to achieve the level of testingrequired Stub-O-Matic utilises the ubiquitous HTTP protocol to receive stub requests and return stubbed responses tothe system under test To use stubs the system under test must be capable of diverting a request that would normallygo to back-end systems How this is best done depends on the architecture of the system under test Implementationsto date have used JAVA AOP Python and ActionalProgress Stub Itineraries
Stubo supports unicode It has not been designed for requests or responses containing bit maps or other non-standardcharacters
1
Stub-O-Matic Documentation Release 0X beta
2 Chapter 1 Introduction
CHAPTER 2
Stub Basics
Stubo simulates back-end systems by matching a request to a response RequestResponse definitions are lsquoloadedrsquo intoStubo
Conventions for this documentation
All Stubo commands are to be prefixed with the server (and port) of the Stubo instance you are using For examplestuboapigetstatus would really be something like lsquohttpmystubostuboapigetstatuslsquo
21 Load and Retrieve Stubs
For the impatient point your browser at stubodefaultexeccmdscmdfile=staticcmdsdemofirstyaml The JSONresponse from Stubo is meant for machines to read For a more human friendly version use the lsquobackrsquo button on yourbrowser and navigate to the lsquoTrackerrsquo page There you will see the most recent Stubo commands and their details Thecommands you just ran from firstyaml are
First use of Stub-O-Matic run this from a browser with uri httpltstubo servergtstubodefaultexeccmdscmdfile=staticcmdsdemofirstyaml
Describe your stubs hererecording
scenario firstsession first_1stubs-json
request method GETbodyPatterns [
contains [get my stub]
]
response
status 200body Hello 1+1 World
3
Stub-O-Matic Documentation Release 0X beta
Provide your requests hereplayback
scenario firstsession first_1requests-
json method GETbody timestamp 092345
get my stub
These will be listed on the Tracker page What just happened The command you made executed a Stub-O-Maticcommand file (demofirstyaml) That file loaded a stub into Stuborsquos database then simulated a request returning theexpected response These actions were logged and displayed on the Tracker page
There are a few key concepts that will make stubbing work
bull Scenario - A re-usable set of stubs
bull Session - The instance or use of the stubs from a scenario
bull Matcher - The matcher used to find the correct response for a request
bull Request - A message normally sent to a back-end by the system under test
bull Response - A message normally returned by a back-end system
bull Stub = Matcher(s) and a response
Stubo commands have been designed to be read written and used by either humans or the Stubo code The stub andcommand files are accessed via a URI (http) In practice this means putting them into a source code repositorysuch as Subversion (SVN) which you should be doing anyway The file location in the execcmds command can belocal to Stubo as in the firstyaml example or (more usual) a URL
For example you could run
stubodefaultexeccmdscmdfile=httpsyour-source-code-repostaticcmdsdemofirstyaml
22 Sessions and Scenarios
In order for stubs to be loaded and retrieved in a predictable automated manner by multiple users many Stubo eventsare controlled by a session A session is a unit of work that must be started and ended Sessions have two possiblemodes of operation
bull record - for loading stubs into the Stubo datastore
bull playback - for retrieving stubbed responses during testing
A scenario is a group of stubs that can be used and re-used by multiple sessions
23 Recording Stubs
Stubs can be created by hand or auto-recorded from a real system under test This is done from what we call a lsquoStubointegrator or Stubo client servicersquo A Stubo integrator is typically a custom library that intercepts real system calls andlsquore-playsrsquo them into a Stubo lsquorecordingrsquo via putstub commands Java and Python Stubo client libraries are provided
4 Chapter 2 Stub Basics
Stub-O-Matic Documentation Release 0X beta
to help integrate Stubo into your applications These clients are maintained in github at httpsgithubcomStub-O-Maticpython-client amp httpsgithubcomStub-O-Maticjava-client
If the real system changes its interface a recording can be repeated to capture the changes Of course this requires thatthe test back-end systems have the correct data in them to be used successfully
Stubo provides mechanisms to deal with context sensitive information such as transaction ids dates user names etcwhich makes recorded stubs less fragile
23 Recording Stubs 5
Stub-O-Matic Documentation Release 0X beta
6 Chapter 2 Stub Basics
CHAPTER 3
Stubo Command File
Command files are used to load stub files and add behaviour to them (state dates etc)
Stubs can also be loaded and tested using individual calls to the Stub-O-Matic server using the Stubo HTTP APICommand files are a shorthand for the API making it simpler to enable repeatable automated interactions with Stub-O-Matic by grouping a set of commands together required for a particular test
31 Command file (YAML)
An example
Stubo YAML Execution ordering is 1) commands 2) recording 3) playback
Commands go herecommands
-putmodule
varsname staticcmdstestsrestyamlnooppy
-putdelay_policy
varsname slowdelay_type fixedmilliseconds 1000
Describe your stubs hererecording
scenario restsession rest_recordingstubs-file stub1jsonvars
recorded_at as_date(2015-01-10)ext_module noop
-json
request method GETbodyPatterns [
7
Stub-O-Matic Documentation Release 0X beta
jsonpath [cmdx]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all ok 2headers Content-Type applicationjsonX-Custom-Header 1234
vars
foo bar
Provide your requests hereplayback
scenario restsession rest_playbackrequests-
file request1jsonvars
played_at as_date(2015-01-20)-
json method GETbody cmd x yheaders
Content-Type applicationjsonX-Custom-Header 1234
varsplayed_at as_date(2015-01-20)
Note the payloads can be defined either in place with the lsquojsonrsquo key or point to an external reference via the lsquofilersquo keyThe file can either reference a local statically served file as shown above or be a URI reference Typically it will be aURI reference to a location in a users source control system where their stubs are stored
The local file references in this example are shown below
stub1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
8 Chapter 3 Stubo Command File
Stub-O-Matic Documentation Release 0X beta
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
request1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
Note that these json payloads for the request and response are defined as strings Stubo also excepts the definition asdictionaries
32 Command Scripting
The YAML file is run through a template processor before being parsed and executed by Stubo Any variables definedsuch as lsquoplayed_atrsquo will evaluated and appropriate subsitutions made
A roll date example
(daterollyaml)
playbackrequests- file dateroll_1433930288_0requestresponse dateroll_1433930288_0stubo_responsevars
getresponse_arg this stub was played at 2015-06-10 095744839438
32 Command Scripting 9
Stub-O-Matic Documentation Release 0X beta
play_date 2014-09-12priority 1putstub_arg this stub was recorded at 2015-06-10 095744839387rec_date 2014-09-10tracking_level full
scenario daterollsession dateroll_1433930288
recordingscenario daterollsession dateroll_1433930288stubs- file dateroll_1433930288_0json
Referenced files
dateroll_1433930288_0json
priority 1args
priority 1rec_date 2014-09-10putstub_arg this stub was recorded at 2015-06-10 095744839387
request
bodyPatterns contains [
ltxml version=10 encoding=UTF-8 standalone=yes gtnltrollmegt n ltOriginDateTimegtroll_date(2014-09-10 as_date(rec_date) as_date(play_date))T000000ZltOriginDateTimegtnltrollmegt]
method POST
response
body ltresponsegtnltputstub_arggt raw putstub_arg ltputstub_arggtnltgetresponse_arggt getresponse_arg ltgetresponse_arggtnltresponsegtstatus 200
dateroll_1433930288_0request
body ltxml version=10 encoding=UTF-8 standalone=yesgtnltrollmegtn ltOriginDateTimegt2014-09-12T000000ZltOriginDateTimegtnltrollmegtheaders host nullpath nullquery uri nullmethod POST
10 Chapter 3 Stubo Command File
CHAPTER 4
Stubo Command File (text format)
The line orientated text version of the command file is an old format It does not support the matching required forREST For example you canrsquot match on the request headers or method It only supports one type of matcher whichis the lsquobody containsrsquo matcher If the text specified for the lsquobody containsrsquo matcher stub is found in the request amatch is found This is often however the only type of matcher required to successfully stub a system This format isstill supported for import via execcmds but will not be exported anymore If a text commands file is imported via alsquocommandsrsquo text file on export via getexport a lsquoyamlrsquo file will be exported
41 Text Command file
bull Lines beginning with a lsquorsquo are treated as comments
bull Commands follow the Stub-O-Matic REST API for example be-ginsessionscenario=abcampsession=12345ampmode=record
bull When the API requires something in the HTTP body input can be sourced by listing files after the command(separated by commas) Files are assumed to be in the same directory as the command file Alternatively a fullURI can be provided to source the input (matcher request or response)
bull Blank lines are ignored
bull An example command is putstubsession=12345rsquofeng_001textMatcher feng_002textMatcherfeng_001response A putstub command must have at least one matcher file and one response file Thisexample has two matchers
What goes in the command files
bull A textMatcher file could be some or all of the text from the request eg AB1234ltFlightInfoQuerygt
bull Multiple textMatchers files may be used in a stub All must evaluate to True against the request for a responseto be returned
bull request files contain the text of a request (eg text xml json) Note the using playback in commands is onlyused to test the stubs as these would be issued by the real system being tested in normal situations
bull response files contain the text of a response (eg text xml json)
bull Any naming convention can be used for matcher request and response files We have used suffixes oftextMatcher request amp response but you are free to use other extensions like matcherxml requestjson re-sponsexml etc
11
Stub-O-Matic Documentation Release 0X beta
42 Command Scripting
Stub-O-Matic supports the ability to load stubs from various and multiple sources This means people can organise allthe stubs that support a particular service or use case together and any test that needs those stubs could pull them infrom common stub libraries maximising stub reuse
set foreign_url = httpwwwgooglecouk set other_url = httpwwwsuncom deletestubsscenario=smartbeginsessionscenario=smartampsession=smart_1ampmode=recordputstubsession=smart_1url=other_urlurl=foreign_urlputstubsession=smart_1text=random_rubbletext=response_textendsessionsession=smart_1
Command files are also programmable with Python code snippets See the example below
set session_name = smart_1 deletestubsscenario=smartbeginsessionscenario=smartampsession=session_nameampmode=recordputstubsession=session_namefirsttextMatcherfirstresponseendsessionsession=session_namebeginsessionscenario=smartampsession=session_nameampmode=playback make 3 requests for i in range(03) getresponsesession=session_namefirstrequest end endsessionsession=session_name
43 Passing Arguments
Arguments can be passed into command files and used in them The following example allows the session and scenarioto be named externally to the command file
execcmdscmdFile=staticcmdstestsacceptsmart_argcommandsampscen=bobampsession=smart_1
The above call can be used as follows
deletestubsscenario=scen[0]beginsessionscenario=scen[0]ampsession= session[0] ampmode=recordputstubsession=session[0]firsttextMatcherfirstresponseendsessionsession=session[0]beginsessionscenario=scen[0]ampsession=session[0]ampmode=playback for i in range(03) getresponsesession=session[0]firstrequest end endsessionsession=session[0]
12 Chapter 4 Stubo Command File (text format)
CHAPTER 5
GUI
51 Tracker
The Stubo lsquoTrackerrsquo page shows details of all requests submitted to Stub-O-Matic and the received response Clickon the link in the function column to show a detailed view Debug level tracking can be added to a getresponse callif lsquotracking_level=fullrsquo is added to the call Matching errors will also trigger some extra debug data to be added to anormal track record
Stubo can not keep tracker data forever so it will eventually drop off the server as space is used up
The tracker page shows all Stubo actions performed on the server Filters are provided to filter on session datetimeresponse times and errors
To go into detail on a particular transaction click the link in the lsquoFunctionrsquo column or lsquoDetailsrsquo in the lsquoStubo Responsersquocolumn For a getresponse call the matchers and responses for the call can be found from the gtgtgt view scenario details(stub matchers and responses) link at the top of the details page
Filtering The tracker filter fields should be self explanatory By default the tracker page shows only data sent to theStubo URL shown at the top of the browser To see tracker items from other URLs on the same hardware tick thelsquoAll Hostsrsquo box
52 Manage
The manage page allows users to execute command files and view system state
The list of loaded stubs includes the date the stubs were loaded into Stubo and the last time the stub was used Clickingthe scenario name shows the stored matchers and responses for the scenario
Also available is a command file scratchpad for entering one or more commands For example one could enter
endsessionsession=first_sessiondeletestubsscenario=firstendsessionsession=second_sessiondeletestubsscenario=second
All the above commands would then be executed See the results on the tracker page
The manage page also shows stubs delay policies and any external modules loaded
13
Stub-O-Matic Documentation Release 0X beta
53 Analytics
The lsquoAnalyticsrsquo tab takes you to a dashboard showing statistics about the Stubo server The orange bar along the topallows you to select different Stubo servers and time periods The first of the 4 squares on the right turns on a legend
Charts
chart title descriptionstubssecondresponse rate
number of stubs requested per second
delay added user defined delays to stubs these can be specified to emulate response times of thesystem being stubbed
latency average time taken to return a stub excluding network and user defined delaysrequestresponse size size in kb of the requests to stubo and responses deliveredhost response latency where Stubo is deployed across multiple servers this chart compares the different servers
against each otherfailure rate percent of failed requests (HTTP response gt= 400)health checks Stubo has a regular health check ping to demonstrate that all systems are working
properlyrunning putstubcount
cumulative total of stubs loaded
running responsecount
cumulative total of stubs retrieved
Also note the Stubo API documentation for getstats
14 Chapter 5 GUI
CHAPTER 6
Integration with Test Tools
Stub-O-Matic works with all major software test execution tools Stubo of course does not run tests or evaluateresults it emulates back-end systems needed by tests
Data dependent software tests typically consist of 3 steps
1 Data setup
2 Test execution
3 Data teardown
Working with Stubo to provide back-end data is no different
Stuborsquos API can be used to setup or teardown data directly Stubo command files offer a shorthand for using the APIand are much more compact Any test tool that can make HTTP calls can use Stubo
61 Test steps example
1 Trigger an http call similar to this from the test tool of your choice
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_setupcommands
2 Run tests
3 Trigger a teardown http call similar to this from the test tool
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_teardowncommands
Setup would typically i) delete stubs ii) set any delays iii) submit a recording via a series of putstub calls and finallyiv) begin a session in playback
Teardown would typically i) end session ii) delete stubs
Setup and teardown can be performed with a mixture of command files (YAML or commands) and direct HTTP APIcalls to stubo
15
Stub-O-Matic Documentation Release 0X beta
16 Chapter 6 Integration with Test Tools
CHAPTER 7
Stubbing
71 Add Delay to a Response
To emulate response times of systems being stubbed one can set a delay on stubs Delays are set on the stub responsenot on the request to stubo This is because one request into the system being tested can result in several back-endstubocalls each of which might need different delay settings
Stubo provides reusable delay_policy objects which can be used across many stubs and across sessions The de-lay_policy can be altered on the fly during a loadperformance test if desired There are currently two delay_policytypes to choose from (1) fixed and (2)random with mean and standard deviation
A fixed delay policy can be created or modified with a commnand such as
putdelay_policyname=a_fixed_delay_policyampdelay_type=fixedampmilliseconds=200
Note that the value for the lsquonamersquo can be set to any string and is used again in the putstub A variable delay can be setup as follows
putdelay_policyname=a_normalvariate_delay_policyampdelay_type=normalvariateampmean=100ampstddev=50
Allowed lsquodelay typersquo values are lsquofixedrsquo and lsquonormalvariatersquo The next step is to use the delay policy when loadingstubs eg
putstubsession=abcampdelay_policy=a_fixed_delay_policy
bull Whenever the stub above is used the named delay policy will be applied
bull Delay policies may be altered on the fly during a test run if desired To change a delay on the fly simply repeatthe putdelay_policy command with a different value Typically this would be done directly by the test toolrather than from a command file
bull Delay policies can be used across multiple sessions This allows for easier alteration of policies where manysessions use the same policy It also means that you should name policies carefully and only alter or sharepolicies you are in control of
72 Matching
Stub-O-Matic currently supports various types of matchers
17
Stub-O-Matic Documentation Release 0X beta
73 Body contains matching
One or more matchers can be defined whereby all matchers must be contained in the request to return a specifiedresponse A typical difficulty with system requests is that they often contain superfluous information such as sessionIDs or time stamps that get in the way of matching a request to the desired response One way Stub-O-Matic can solvethis problem is by the use of lsquomatchersrsquo Matchers are parts of the request that matter to you only what is needed tofind the correct response file
For example a request may include
ltsessionidgt123456ltsessionidgtlttransactionidgt0987654321-9999lttransactionidgt
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Of these 4 lines only the last 2 matter the first 2 lines will differ from one request to the next and be set by the systemmaking the request This request then will be different each time it is made How can one easily match a request likethis to a canned response file without writing code for each requestresponse pair The solution used by Stub-O-Maticis to not store requestresponse file pairs but rather store matchers and the response In the case above the matcherswould be
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Stub-O-Matic contains code which will take a request as input search for the response with the best matchers Everymatcher must be contained in the request to be a match When a match is found the corresponding response is servedback
All matchers must be present in the request to make a match Stub-O-Matic will return the response from the firstmatch if there is more than one possible match Note that whitespace and carriage returns are ignored when matching
Alternatives to removing parts of the request you donrsquot want to match on are
1 using a matcher template to substitute ids or dates etc from the request into the matcher OR
2 using a user exit to run custom code A convenience class XMLManglerExit is provided for XML payloads toignore XMl elements andor attributes from the matching process
74 Templated Matcher
This is an example of a template matcher than extracts values from the request into the matcher The xmltree object isan lxml parsed root element passed into the template
ltrequestgtlttransidgtxmltreexpath(requesttransid)[0]textlttransidgtltdtgt2009-10-24T000000+0000ltdtgtltflightnumbergt0455ltflightnumbergtltrequestgt
75 Templated Responses
A hand crafted Stubo response may contain variables which are substituted in For example a response may contain
set flight_nbr=XX1234 The flight number is flight_nbr
18 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
The resulting response served wil be
The flight number is XX1234
Code may be used for example adding random elements to a response
import random set rnd_nbr = randomrandint(1200) Hello World 1000 + rnd_nbr
The resulting response served will be
Hello World - followed by a number between 1000 and 1200
To put todayrsquos date in a response include the following
today_strformat(dmy)
Stubo responses are run through a Tornado template and any logic or commands allowed in these templates can beused See httpwwwtornadoweborgenstabletemplatehtml for details
76 Request Data in Responses
761 Text Manipulation
It is possible to pull data from the stub request and put it in the response Say you have 200 stubs that are identicalapart from a userid They could be condensed into one stub which uses the userid from the request in the responserequest
ltuseridgtabc123ltuseridgt
matcher
ltuseridgt
response template (using Python)
Hello to request_text[814]
response
Hello to abc123
762 XML Manipulation
Sometimes it is easier to use XPATH to pull data from a request and place it in a response If you need the SessionIdfrom this request
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtJohn1234ltwbsSessionIdgt
Pull the SessionId from the request and use it within the response template
76 Request Data in Responses 19
Stub-O-Matic Documentation Release 0X beta
set namespaces = soapenv httpschemasxmlsoaporgsoapenvelopewbs httpxmlaaacomws200901WBS_Session-20xsd
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtxmltreexpath(soapenvHeaderwbsSessionwbsSessionId
namespaces=namespaces)[0]textltwbsSessionIdgt
Note the xmltree variable is the parsed xml request (as an lxml Root object) made available to the template if therequest is valid xml
Another example - no namespaces in the request Request
ltxml version=10 encoding=UTF-8gtltCompensateCustomersCheck APCWCustom=-1 LocalCurrencyCode=GBP NumberOfAdults=1gt
Response
set currency_code = xmltreexpath(CompensateCustomersCheck)[0]attrib[LocalCurrencyCode] ltresponse LocalCurrencyCode=currency_codegtpay meltresponsegt
77 Stateful Stubs
State is important in simulating back-end systems For example if one was to check-in passenger Bob on a particularflight the back-end would allow this request to succeed only one time and subsequent attempts to check-in the samepassenger on the same flight would be rejected Stubo can emulate this functionality or if desired for repeatable testsStubo can allow Bob to be checked in multiple times on the same flight
Stateful stubbing is enabled automatically If stubs are recorded or loaded in which the same request returns differentresponses Stubo will remember this and internally keep track of which response should come next
If your test is to check-in Bob multiple times simply do not load the stubs representing the check-in rejection andStubo will allow multiple check-ins
78 User Exits
Stub-O-Matic provides hooks into the runtime to execute custom user code Stubo can execute your custom code totransform stub matchers requests andor responses This is a powerful mechanism when your stubs contain dynamicdata or need to change over time This can be useful to perform lsquointelligent stubbingrsquo for example to roll dates Theuser exit API is contained in the stuboextuser_exit module
781 API
782 XML Auto Mangling
If your data is XML a convenience class XMLUserExit is provided to enable auto mangling
20 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
783 API
79 Hooks
710 Modules
Modules are added with the putmodule command and referenced via the ext_module url arg in the putstub call Forexample
putmodulename=staticcmdstestsextxsltmanglerpydeletestubsscenario=mangler_xsltbeginsessionscenario=mangler_xsltampsession=mangler_xslt1ampmode=recordputstubsession=mangler_xslt1ampext_module=mangler firstrequest firstresponseendsessionsession=mangler_xslt1
This example dynamically imports a test mangler from the Stubo server Typically these external python modules thathook into the Stubo runtime would be sourced from source control system using a URL
Note user exit processing is only enabled if the lsquoext_modulersquo variable is set to a pre-loaded module during the putstubcall Each Stub-O-Matic host URL has its own copy of any loaded modules Such modules are available to any testand as such should typically be loaded once per test suite
711 Splitting
Useful to removing random elements from matchers It can happen that the system under test will include seeminglyrandom elements in the request This will cause the next run of the test to not match This problem can be solved byprocessing matchers when loading - during a record A matcher can be split into two with the variable text left outFuture request must match both of the matchers to get the response An example is
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N11111234567806P1ltCommandgtltCryptic_GetScreen_Querygt
Which becomes two matchers
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N1111
and
06P1ltCommandgtltCryptic_GetScreen_Querygt
cutting out the random number lsquo12345678rsquo in the middle
This matcher splitting can be done manually or automatically with a user exit Seestaticcmdstestsextsplitters1firstcommands for and example It uses the rules module at staticcmdstestsextaaapy
Multiple matchers excluding random request elements can be created manually and loaded as such
putstubsession=splitter_record splitter_1textMatcher splitter_2textMatcher splitterresponse
712 Caching Values
If emulating back-end behaviour means writing some code for a particularly tricky behaviour Stubo exposes a simplekey-value cache API to matchers and responses
79 Hooks 21
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
Stub-O-Matic Documentation Release 0X beta
2 Chapter 1 Introduction
CHAPTER 2
Stub Basics
Stubo simulates back-end systems by matching a request to a response RequestResponse definitions are lsquoloadedrsquo intoStubo
Conventions for this documentation
All Stubo commands are to be prefixed with the server (and port) of the Stubo instance you are using For examplestuboapigetstatus would really be something like lsquohttpmystubostuboapigetstatuslsquo
21 Load and Retrieve Stubs
For the impatient point your browser at stubodefaultexeccmdscmdfile=staticcmdsdemofirstyaml The JSONresponse from Stubo is meant for machines to read For a more human friendly version use the lsquobackrsquo button on yourbrowser and navigate to the lsquoTrackerrsquo page There you will see the most recent Stubo commands and their details Thecommands you just ran from firstyaml are
First use of Stub-O-Matic run this from a browser with uri httpltstubo servergtstubodefaultexeccmdscmdfile=staticcmdsdemofirstyaml
Describe your stubs hererecording
scenario firstsession first_1stubs-json
request method GETbodyPatterns [
contains [get my stub]
]
response
status 200body Hello 1+1 World
3
Stub-O-Matic Documentation Release 0X beta
Provide your requests hereplayback
scenario firstsession first_1requests-
json method GETbody timestamp 092345
get my stub
These will be listed on the Tracker page What just happened The command you made executed a Stub-O-Maticcommand file (demofirstyaml) That file loaded a stub into Stuborsquos database then simulated a request returning theexpected response These actions were logged and displayed on the Tracker page
There are a few key concepts that will make stubbing work
bull Scenario - A re-usable set of stubs
bull Session - The instance or use of the stubs from a scenario
bull Matcher - The matcher used to find the correct response for a request
bull Request - A message normally sent to a back-end by the system under test
bull Response - A message normally returned by a back-end system
bull Stub = Matcher(s) and a response
Stubo commands have been designed to be read written and used by either humans or the Stubo code The stub andcommand files are accessed via a URI (http) In practice this means putting them into a source code repositorysuch as Subversion (SVN) which you should be doing anyway The file location in the execcmds command can belocal to Stubo as in the firstyaml example or (more usual) a URL
For example you could run
stubodefaultexeccmdscmdfile=httpsyour-source-code-repostaticcmdsdemofirstyaml
22 Sessions and Scenarios
In order for stubs to be loaded and retrieved in a predictable automated manner by multiple users many Stubo eventsare controlled by a session A session is a unit of work that must be started and ended Sessions have two possiblemodes of operation
bull record - for loading stubs into the Stubo datastore
bull playback - for retrieving stubbed responses during testing
A scenario is a group of stubs that can be used and re-used by multiple sessions
23 Recording Stubs
Stubs can be created by hand or auto-recorded from a real system under test This is done from what we call a lsquoStubointegrator or Stubo client servicersquo A Stubo integrator is typically a custom library that intercepts real system calls andlsquore-playsrsquo them into a Stubo lsquorecordingrsquo via putstub commands Java and Python Stubo client libraries are provided
4 Chapter 2 Stub Basics
Stub-O-Matic Documentation Release 0X beta
to help integrate Stubo into your applications These clients are maintained in github at httpsgithubcomStub-O-Maticpython-client amp httpsgithubcomStub-O-Maticjava-client
If the real system changes its interface a recording can be repeated to capture the changes Of course this requires thatthe test back-end systems have the correct data in them to be used successfully
Stubo provides mechanisms to deal with context sensitive information such as transaction ids dates user names etcwhich makes recorded stubs less fragile
23 Recording Stubs 5
Stub-O-Matic Documentation Release 0X beta
6 Chapter 2 Stub Basics
CHAPTER 3
Stubo Command File
Command files are used to load stub files and add behaviour to them (state dates etc)
Stubs can also be loaded and tested using individual calls to the Stub-O-Matic server using the Stubo HTTP APICommand files are a shorthand for the API making it simpler to enable repeatable automated interactions with Stub-O-Matic by grouping a set of commands together required for a particular test
31 Command file (YAML)
An example
Stubo YAML Execution ordering is 1) commands 2) recording 3) playback
Commands go herecommands
-putmodule
varsname staticcmdstestsrestyamlnooppy
-putdelay_policy
varsname slowdelay_type fixedmilliseconds 1000
Describe your stubs hererecording
scenario restsession rest_recordingstubs-file stub1jsonvars
recorded_at as_date(2015-01-10)ext_module noop
-json
request method GETbodyPatterns [
7
Stub-O-Matic Documentation Release 0X beta
jsonpath [cmdx]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all ok 2headers Content-Type applicationjsonX-Custom-Header 1234
vars
foo bar
Provide your requests hereplayback
scenario restsession rest_playbackrequests-
file request1jsonvars
played_at as_date(2015-01-20)-
json method GETbody cmd x yheaders
Content-Type applicationjsonX-Custom-Header 1234
varsplayed_at as_date(2015-01-20)
Note the payloads can be defined either in place with the lsquojsonrsquo key or point to an external reference via the lsquofilersquo keyThe file can either reference a local statically served file as shown above or be a URI reference Typically it will be aURI reference to a location in a users source control system where their stubs are stored
The local file references in this example are shown below
stub1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
8 Chapter 3 Stubo Command File
Stub-O-Matic Documentation Release 0X beta
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
request1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
Note that these json payloads for the request and response are defined as strings Stubo also excepts the definition asdictionaries
32 Command Scripting
The YAML file is run through a template processor before being parsed and executed by Stubo Any variables definedsuch as lsquoplayed_atrsquo will evaluated and appropriate subsitutions made
A roll date example
(daterollyaml)
playbackrequests- file dateroll_1433930288_0requestresponse dateroll_1433930288_0stubo_responsevars
getresponse_arg this stub was played at 2015-06-10 095744839438
32 Command Scripting 9
Stub-O-Matic Documentation Release 0X beta
play_date 2014-09-12priority 1putstub_arg this stub was recorded at 2015-06-10 095744839387rec_date 2014-09-10tracking_level full
scenario daterollsession dateroll_1433930288
recordingscenario daterollsession dateroll_1433930288stubs- file dateroll_1433930288_0json
Referenced files
dateroll_1433930288_0json
priority 1args
priority 1rec_date 2014-09-10putstub_arg this stub was recorded at 2015-06-10 095744839387
request
bodyPatterns contains [
ltxml version=10 encoding=UTF-8 standalone=yes gtnltrollmegt n ltOriginDateTimegtroll_date(2014-09-10 as_date(rec_date) as_date(play_date))T000000ZltOriginDateTimegtnltrollmegt]
method POST
response
body ltresponsegtnltputstub_arggt raw putstub_arg ltputstub_arggtnltgetresponse_arggt getresponse_arg ltgetresponse_arggtnltresponsegtstatus 200
dateroll_1433930288_0request
body ltxml version=10 encoding=UTF-8 standalone=yesgtnltrollmegtn ltOriginDateTimegt2014-09-12T000000ZltOriginDateTimegtnltrollmegtheaders host nullpath nullquery uri nullmethod POST
10 Chapter 3 Stubo Command File
CHAPTER 4
Stubo Command File (text format)
The line orientated text version of the command file is an old format It does not support the matching required forREST For example you canrsquot match on the request headers or method It only supports one type of matcher whichis the lsquobody containsrsquo matcher If the text specified for the lsquobody containsrsquo matcher stub is found in the request amatch is found This is often however the only type of matcher required to successfully stub a system This format isstill supported for import via execcmds but will not be exported anymore If a text commands file is imported via alsquocommandsrsquo text file on export via getexport a lsquoyamlrsquo file will be exported
41 Text Command file
bull Lines beginning with a lsquorsquo are treated as comments
bull Commands follow the Stub-O-Matic REST API for example be-ginsessionscenario=abcampsession=12345ampmode=record
bull When the API requires something in the HTTP body input can be sourced by listing files after the command(separated by commas) Files are assumed to be in the same directory as the command file Alternatively a fullURI can be provided to source the input (matcher request or response)
bull Blank lines are ignored
bull An example command is putstubsession=12345rsquofeng_001textMatcher feng_002textMatcherfeng_001response A putstub command must have at least one matcher file and one response file Thisexample has two matchers
What goes in the command files
bull A textMatcher file could be some or all of the text from the request eg AB1234ltFlightInfoQuerygt
bull Multiple textMatchers files may be used in a stub All must evaluate to True against the request for a responseto be returned
bull request files contain the text of a request (eg text xml json) Note the using playback in commands is onlyused to test the stubs as these would be issued by the real system being tested in normal situations
bull response files contain the text of a response (eg text xml json)
bull Any naming convention can be used for matcher request and response files We have used suffixes oftextMatcher request amp response but you are free to use other extensions like matcherxml requestjson re-sponsexml etc
11
Stub-O-Matic Documentation Release 0X beta
42 Command Scripting
Stub-O-Matic supports the ability to load stubs from various and multiple sources This means people can organise allthe stubs that support a particular service or use case together and any test that needs those stubs could pull them infrom common stub libraries maximising stub reuse
set foreign_url = httpwwwgooglecouk set other_url = httpwwwsuncom deletestubsscenario=smartbeginsessionscenario=smartampsession=smart_1ampmode=recordputstubsession=smart_1url=other_urlurl=foreign_urlputstubsession=smart_1text=random_rubbletext=response_textendsessionsession=smart_1
Command files are also programmable with Python code snippets See the example below
set session_name = smart_1 deletestubsscenario=smartbeginsessionscenario=smartampsession=session_nameampmode=recordputstubsession=session_namefirsttextMatcherfirstresponseendsessionsession=session_namebeginsessionscenario=smartampsession=session_nameampmode=playback make 3 requests for i in range(03) getresponsesession=session_namefirstrequest end endsessionsession=session_name
43 Passing Arguments
Arguments can be passed into command files and used in them The following example allows the session and scenarioto be named externally to the command file
execcmdscmdFile=staticcmdstestsacceptsmart_argcommandsampscen=bobampsession=smart_1
The above call can be used as follows
deletestubsscenario=scen[0]beginsessionscenario=scen[0]ampsession= session[0] ampmode=recordputstubsession=session[0]firsttextMatcherfirstresponseendsessionsession=session[0]beginsessionscenario=scen[0]ampsession=session[0]ampmode=playback for i in range(03) getresponsesession=session[0]firstrequest end endsessionsession=session[0]
12 Chapter 4 Stubo Command File (text format)
CHAPTER 5
GUI
51 Tracker
The Stubo lsquoTrackerrsquo page shows details of all requests submitted to Stub-O-Matic and the received response Clickon the link in the function column to show a detailed view Debug level tracking can be added to a getresponse callif lsquotracking_level=fullrsquo is added to the call Matching errors will also trigger some extra debug data to be added to anormal track record
Stubo can not keep tracker data forever so it will eventually drop off the server as space is used up
The tracker page shows all Stubo actions performed on the server Filters are provided to filter on session datetimeresponse times and errors
To go into detail on a particular transaction click the link in the lsquoFunctionrsquo column or lsquoDetailsrsquo in the lsquoStubo Responsersquocolumn For a getresponse call the matchers and responses for the call can be found from the gtgtgt view scenario details(stub matchers and responses) link at the top of the details page
Filtering The tracker filter fields should be self explanatory By default the tracker page shows only data sent to theStubo URL shown at the top of the browser To see tracker items from other URLs on the same hardware tick thelsquoAll Hostsrsquo box
52 Manage
The manage page allows users to execute command files and view system state
The list of loaded stubs includes the date the stubs were loaded into Stubo and the last time the stub was used Clickingthe scenario name shows the stored matchers and responses for the scenario
Also available is a command file scratchpad for entering one or more commands For example one could enter
endsessionsession=first_sessiondeletestubsscenario=firstendsessionsession=second_sessiondeletestubsscenario=second
All the above commands would then be executed See the results on the tracker page
The manage page also shows stubs delay policies and any external modules loaded
13
Stub-O-Matic Documentation Release 0X beta
53 Analytics
The lsquoAnalyticsrsquo tab takes you to a dashboard showing statistics about the Stubo server The orange bar along the topallows you to select different Stubo servers and time periods The first of the 4 squares on the right turns on a legend
Charts
chart title descriptionstubssecondresponse rate
number of stubs requested per second
delay added user defined delays to stubs these can be specified to emulate response times of thesystem being stubbed
latency average time taken to return a stub excluding network and user defined delaysrequestresponse size size in kb of the requests to stubo and responses deliveredhost response latency where Stubo is deployed across multiple servers this chart compares the different servers
against each otherfailure rate percent of failed requests (HTTP response gt= 400)health checks Stubo has a regular health check ping to demonstrate that all systems are working
properlyrunning putstubcount
cumulative total of stubs loaded
running responsecount
cumulative total of stubs retrieved
Also note the Stubo API documentation for getstats
14 Chapter 5 GUI
CHAPTER 6
Integration with Test Tools
Stub-O-Matic works with all major software test execution tools Stubo of course does not run tests or evaluateresults it emulates back-end systems needed by tests
Data dependent software tests typically consist of 3 steps
1 Data setup
2 Test execution
3 Data teardown
Working with Stubo to provide back-end data is no different
Stuborsquos API can be used to setup or teardown data directly Stubo command files offer a shorthand for using the APIand are much more compact Any test tool that can make HTTP calls can use Stubo
61 Test steps example
1 Trigger an http call similar to this from the test tool of your choice
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_setupcommands
2 Run tests
3 Trigger a teardown http call similar to this from the test tool
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_teardowncommands
Setup would typically i) delete stubs ii) set any delays iii) submit a recording via a series of putstub calls and finallyiv) begin a session in playback
Teardown would typically i) end session ii) delete stubs
Setup and teardown can be performed with a mixture of command files (YAML or commands) and direct HTTP APIcalls to stubo
15
Stub-O-Matic Documentation Release 0X beta
16 Chapter 6 Integration with Test Tools
CHAPTER 7
Stubbing
71 Add Delay to a Response
To emulate response times of systems being stubbed one can set a delay on stubs Delays are set on the stub responsenot on the request to stubo This is because one request into the system being tested can result in several back-endstubocalls each of which might need different delay settings
Stubo provides reusable delay_policy objects which can be used across many stubs and across sessions The de-lay_policy can be altered on the fly during a loadperformance test if desired There are currently two delay_policytypes to choose from (1) fixed and (2)random with mean and standard deviation
A fixed delay policy can be created or modified with a commnand such as
putdelay_policyname=a_fixed_delay_policyampdelay_type=fixedampmilliseconds=200
Note that the value for the lsquonamersquo can be set to any string and is used again in the putstub A variable delay can be setup as follows
putdelay_policyname=a_normalvariate_delay_policyampdelay_type=normalvariateampmean=100ampstddev=50
Allowed lsquodelay typersquo values are lsquofixedrsquo and lsquonormalvariatersquo The next step is to use the delay policy when loadingstubs eg
putstubsession=abcampdelay_policy=a_fixed_delay_policy
bull Whenever the stub above is used the named delay policy will be applied
bull Delay policies may be altered on the fly during a test run if desired To change a delay on the fly simply repeatthe putdelay_policy command with a different value Typically this would be done directly by the test toolrather than from a command file
bull Delay policies can be used across multiple sessions This allows for easier alteration of policies where manysessions use the same policy It also means that you should name policies carefully and only alter or sharepolicies you are in control of
72 Matching
Stub-O-Matic currently supports various types of matchers
17
Stub-O-Matic Documentation Release 0X beta
73 Body contains matching
One or more matchers can be defined whereby all matchers must be contained in the request to return a specifiedresponse A typical difficulty with system requests is that they often contain superfluous information such as sessionIDs or time stamps that get in the way of matching a request to the desired response One way Stub-O-Matic can solvethis problem is by the use of lsquomatchersrsquo Matchers are parts of the request that matter to you only what is needed tofind the correct response file
For example a request may include
ltsessionidgt123456ltsessionidgtlttransactionidgt0987654321-9999lttransactionidgt
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Of these 4 lines only the last 2 matter the first 2 lines will differ from one request to the next and be set by the systemmaking the request This request then will be different each time it is made How can one easily match a request likethis to a canned response file without writing code for each requestresponse pair The solution used by Stub-O-Maticis to not store requestresponse file pairs but rather store matchers and the response In the case above the matcherswould be
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Stub-O-Matic contains code which will take a request as input search for the response with the best matchers Everymatcher must be contained in the request to be a match When a match is found the corresponding response is servedback
All matchers must be present in the request to make a match Stub-O-Matic will return the response from the firstmatch if there is more than one possible match Note that whitespace and carriage returns are ignored when matching
Alternatives to removing parts of the request you donrsquot want to match on are
1 using a matcher template to substitute ids or dates etc from the request into the matcher OR
2 using a user exit to run custom code A convenience class XMLManglerExit is provided for XML payloads toignore XMl elements andor attributes from the matching process
74 Templated Matcher
This is an example of a template matcher than extracts values from the request into the matcher The xmltree object isan lxml parsed root element passed into the template
ltrequestgtlttransidgtxmltreexpath(requesttransid)[0]textlttransidgtltdtgt2009-10-24T000000+0000ltdtgtltflightnumbergt0455ltflightnumbergtltrequestgt
75 Templated Responses
A hand crafted Stubo response may contain variables which are substituted in For example a response may contain
set flight_nbr=XX1234 The flight number is flight_nbr
18 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
The resulting response served wil be
The flight number is XX1234
Code may be used for example adding random elements to a response
import random set rnd_nbr = randomrandint(1200) Hello World 1000 + rnd_nbr
The resulting response served will be
Hello World - followed by a number between 1000 and 1200
To put todayrsquos date in a response include the following
today_strformat(dmy)
Stubo responses are run through a Tornado template and any logic or commands allowed in these templates can beused See httpwwwtornadoweborgenstabletemplatehtml for details
76 Request Data in Responses
761 Text Manipulation
It is possible to pull data from the stub request and put it in the response Say you have 200 stubs that are identicalapart from a userid They could be condensed into one stub which uses the userid from the request in the responserequest
ltuseridgtabc123ltuseridgt
matcher
ltuseridgt
response template (using Python)
Hello to request_text[814]
response
Hello to abc123
762 XML Manipulation
Sometimes it is easier to use XPATH to pull data from a request and place it in a response If you need the SessionIdfrom this request
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtJohn1234ltwbsSessionIdgt
Pull the SessionId from the request and use it within the response template
76 Request Data in Responses 19
Stub-O-Matic Documentation Release 0X beta
set namespaces = soapenv httpschemasxmlsoaporgsoapenvelopewbs httpxmlaaacomws200901WBS_Session-20xsd
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtxmltreexpath(soapenvHeaderwbsSessionwbsSessionId
namespaces=namespaces)[0]textltwbsSessionIdgt
Note the xmltree variable is the parsed xml request (as an lxml Root object) made available to the template if therequest is valid xml
Another example - no namespaces in the request Request
ltxml version=10 encoding=UTF-8gtltCompensateCustomersCheck APCWCustom=-1 LocalCurrencyCode=GBP NumberOfAdults=1gt
Response
set currency_code = xmltreexpath(CompensateCustomersCheck)[0]attrib[LocalCurrencyCode] ltresponse LocalCurrencyCode=currency_codegtpay meltresponsegt
77 Stateful Stubs
State is important in simulating back-end systems For example if one was to check-in passenger Bob on a particularflight the back-end would allow this request to succeed only one time and subsequent attempts to check-in the samepassenger on the same flight would be rejected Stubo can emulate this functionality or if desired for repeatable testsStubo can allow Bob to be checked in multiple times on the same flight
Stateful stubbing is enabled automatically If stubs are recorded or loaded in which the same request returns differentresponses Stubo will remember this and internally keep track of which response should come next
If your test is to check-in Bob multiple times simply do not load the stubs representing the check-in rejection andStubo will allow multiple check-ins
78 User Exits
Stub-O-Matic provides hooks into the runtime to execute custom user code Stubo can execute your custom code totransform stub matchers requests andor responses This is a powerful mechanism when your stubs contain dynamicdata or need to change over time This can be useful to perform lsquointelligent stubbingrsquo for example to roll dates Theuser exit API is contained in the stuboextuser_exit module
781 API
782 XML Auto Mangling
If your data is XML a convenience class XMLUserExit is provided to enable auto mangling
20 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
783 API
79 Hooks
710 Modules
Modules are added with the putmodule command and referenced via the ext_module url arg in the putstub call Forexample
putmodulename=staticcmdstestsextxsltmanglerpydeletestubsscenario=mangler_xsltbeginsessionscenario=mangler_xsltampsession=mangler_xslt1ampmode=recordputstubsession=mangler_xslt1ampext_module=mangler firstrequest firstresponseendsessionsession=mangler_xslt1
This example dynamically imports a test mangler from the Stubo server Typically these external python modules thathook into the Stubo runtime would be sourced from source control system using a URL
Note user exit processing is only enabled if the lsquoext_modulersquo variable is set to a pre-loaded module during the putstubcall Each Stub-O-Matic host URL has its own copy of any loaded modules Such modules are available to any testand as such should typically be loaded once per test suite
711 Splitting
Useful to removing random elements from matchers It can happen that the system under test will include seeminglyrandom elements in the request This will cause the next run of the test to not match This problem can be solved byprocessing matchers when loading - during a record A matcher can be split into two with the variable text left outFuture request must match both of the matchers to get the response An example is
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N11111234567806P1ltCommandgtltCryptic_GetScreen_Querygt
Which becomes two matchers
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N1111
and
06P1ltCommandgtltCryptic_GetScreen_Querygt
cutting out the random number lsquo12345678rsquo in the middle
This matcher splitting can be done manually or automatically with a user exit Seestaticcmdstestsextsplitters1firstcommands for and example It uses the rules module at staticcmdstestsextaaapy
Multiple matchers excluding random request elements can be created manually and loaded as such
putstubsession=splitter_record splitter_1textMatcher splitter_2textMatcher splitterresponse
712 Caching Values
If emulating back-end behaviour means writing some code for a particularly tricky behaviour Stubo exposes a simplekey-value cache API to matchers and responses
79 Hooks 21
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
CHAPTER 2
Stub Basics
Stubo simulates back-end systems by matching a request to a response RequestResponse definitions are lsquoloadedrsquo intoStubo
Conventions for this documentation
All Stubo commands are to be prefixed with the server (and port) of the Stubo instance you are using For examplestuboapigetstatus would really be something like lsquohttpmystubostuboapigetstatuslsquo
21 Load and Retrieve Stubs
For the impatient point your browser at stubodefaultexeccmdscmdfile=staticcmdsdemofirstyaml The JSONresponse from Stubo is meant for machines to read For a more human friendly version use the lsquobackrsquo button on yourbrowser and navigate to the lsquoTrackerrsquo page There you will see the most recent Stubo commands and their details Thecommands you just ran from firstyaml are
First use of Stub-O-Matic run this from a browser with uri httpltstubo servergtstubodefaultexeccmdscmdfile=staticcmdsdemofirstyaml
Describe your stubs hererecording
scenario firstsession first_1stubs-json
request method GETbodyPatterns [
contains [get my stub]
]
response
status 200body Hello 1+1 World
3
Stub-O-Matic Documentation Release 0X beta
Provide your requests hereplayback
scenario firstsession first_1requests-
json method GETbody timestamp 092345
get my stub
These will be listed on the Tracker page What just happened The command you made executed a Stub-O-Maticcommand file (demofirstyaml) That file loaded a stub into Stuborsquos database then simulated a request returning theexpected response These actions were logged and displayed on the Tracker page
There are a few key concepts that will make stubbing work
bull Scenario - A re-usable set of stubs
bull Session - The instance or use of the stubs from a scenario
bull Matcher - The matcher used to find the correct response for a request
bull Request - A message normally sent to a back-end by the system under test
bull Response - A message normally returned by a back-end system
bull Stub = Matcher(s) and a response
Stubo commands have been designed to be read written and used by either humans or the Stubo code The stub andcommand files are accessed via a URI (http) In practice this means putting them into a source code repositorysuch as Subversion (SVN) which you should be doing anyway The file location in the execcmds command can belocal to Stubo as in the firstyaml example or (more usual) a URL
For example you could run
stubodefaultexeccmdscmdfile=httpsyour-source-code-repostaticcmdsdemofirstyaml
22 Sessions and Scenarios
In order for stubs to be loaded and retrieved in a predictable automated manner by multiple users many Stubo eventsare controlled by a session A session is a unit of work that must be started and ended Sessions have two possiblemodes of operation
bull record - for loading stubs into the Stubo datastore
bull playback - for retrieving stubbed responses during testing
A scenario is a group of stubs that can be used and re-used by multiple sessions
23 Recording Stubs
Stubs can be created by hand or auto-recorded from a real system under test This is done from what we call a lsquoStubointegrator or Stubo client servicersquo A Stubo integrator is typically a custom library that intercepts real system calls andlsquore-playsrsquo them into a Stubo lsquorecordingrsquo via putstub commands Java and Python Stubo client libraries are provided
4 Chapter 2 Stub Basics
Stub-O-Matic Documentation Release 0X beta
to help integrate Stubo into your applications These clients are maintained in github at httpsgithubcomStub-O-Maticpython-client amp httpsgithubcomStub-O-Maticjava-client
If the real system changes its interface a recording can be repeated to capture the changes Of course this requires thatthe test back-end systems have the correct data in them to be used successfully
Stubo provides mechanisms to deal with context sensitive information such as transaction ids dates user names etcwhich makes recorded stubs less fragile
23 Recording Stubs 5
Stub-O-Matic Documentation Release 0X beta
6 Chapter 2 Stub Basics
CHAPTER 3
Stubo Command File
Command files are used to load stub files and add behaviour to them (state dates etc)
Stubs can also be loaded and tested using individual calls to the Stub-O-Matic server using the Stubo HTTP APICommand files are a shorthand for the API making it simpler to enable repeatable automated interactions with Stub-O-Matic by grouping a set of commands together required for a particular test
31 Command file (YAML)
An example
Stubo YAML Execution ordering is 1) commands 2) recording 3) playback
Commands go herecommands
-putmodule
varsname staticcmdstestsrestyamlnooppy
-putdelay_policy
varsname slowdelay_type fixedmilliseconds 1000
Describe your stubs hererecording
scenario restsession rest_recordingstubs-file stub1jsonvars
recorded_at as_date(2015-01-10)ext_module noop
-json
request method GETbodyPatterns [
7
Stub-O-Matic Documentation Release 0X beta
jsonpath [cmdx]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all ok 2headers Content-Type applicationjsonX-Custom-Header 1234
vars
foo bar
Provide your requests hereplayback
scenario restsession rest_playbackrequests-
file request1jsonvars
played_at as_date(2015-01-20)-
json method GETbody cmd x yheaders
Content-Type applicationjsonX-Custom-Header 1234
varsplayed_at as_date(2015-01-20)
Note the payloads can be defined either in place with the lsquojsonrsquo key or point to an external reference via the lsquofilersquo keyThe file can either reference a local statically served file as shown above or be a URI reference Typically it will be aURI reference to a location in a users source control system where their stubs are stored
The local file references in this example are shown below
stub1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
8 Chapter 3 Stubo Command File
Stub-O-Matic Documentation Release 0X beta
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
request1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
Note that these json payloads for the request and response are defined as strings Stubo also excepts the definition asdictionaries
32 Command Scripting
The YAML file is run through a template processor before being parsed and executed by Stubo Any variables definedsuch as lsquoplayed_atrsquo will evaluated and appropriate subsitutions made
A roll date example
(daterollyaml)
playbackrequests- file dateroll_1433930288_0requestresponse dateroll_1433930288_0stubo_responsevars
getresponse_arg this stub was played at 2015-06-10 095744839438
32 Command Scripting 9
Stub-O-Matic Documentation Release 0X beta
play_date 2014-09-12priority 1putstub_arg this stub was recorded at 2015-06-10 095744839387rec_date 2014-09-10tracking_level full
scenario daterollsession dateroll_1433930288
recordingscenario daterollsession dateroll_1433930288stubs- file dateroll_1433930288_0json
Referenced files
dateroll_1433930288_0json
priority 1args
priority 1rec_date 2014-09-10putstub_arg this stub was recorded at 2015-06-10 095744839387
request
bodyPatterns contains [
ltxml version=10 encoding=UTF-8 standalone=yes gtnltrollmegt n ltOriginDateTimegtroll_date(2014-09-10 as_date(rec_date) as_date(play_date))T000000ZltOriginDateTimegtnltrollmegt]
method POST
response
body ltresponsegtnltputstub_arggt raw putstub_arg ltputstub_arggtnltgetresponse_arggt getresponse_arg ltgetresponse_arggtnltresponsegtstatus 200
dateroll_1433930288_0request
body ltxml version=10 encoding=UTF-8 standalone=yesgtnltrollmegtn ltOriginDateTimegt2014-09-12T000000ZltOriginDateTimegtnltrollmegtheaders host nullpath nullquery uri nullmethod POST
10 Chapter 3 Stubo Command File
CHAPTER 4
Stubo Command File (text format)
The line orientated text version of the command file is an old format It does not support the matching required forREST For example you canrsquot match on the request headers or method It only supports one type of matcher whichis the lsquobody containsrsquo matcher If the text specified for the lsquobody containsrsquo matcher stub is found in the request amatch is found This is often however the only type of matcher required to successfully stub a system This format isstill supported for import via execcmds but will not be exported anymore If a text commands file is imported via alsquocommandsrsquo text file on export via getexport a lsquoyamlrsquo file will be exported
41 Text Command file
bull Lines beginning with a lsquorsquo are treated as comments
bull Commands follow the Stub-O-Matic REST API for example be-ginsessionscenario=abcampsession=12345ampmode=record
bull When the API requires something in the HTTP body input can be sourced by listing files after the command(separated by commas) Files are assumed to be in the same directory as the command file Alternatively a fullURI can be provided to source the input (matcher request or response)
bull Blank lines are ignored
bull An example command is putstubsession=12345rsquofeng_001textMatcher feng_002textMatcherfeng_001response A putstub command must have at least one matcher file and one response file Thisexample has two matchers
What goes in the command files
bull A textMatcher file could be some or all of the text from the request eg AB1234ltFlightInfoQuerygt
bull Multiple textMatchers files may be used in a stub All must evaluate to True against the request for a responseto be returned
bull request files contain the text of a request (eg text xml json) Note the using playback in commands is onlyused to test the stubs as these would be issued by the real system being tested in normal situations
bull response files contain the text of a response (eg text xml json)
bull Any naming convention can be used for matcher request and response files We have used suffixes oftextMatcher request amp response but you are free to use other extensions like matcherxml requestjson re-sponsexml etc
11
Stub-O-Matic Documentation Release 0X beta
42 Command Scripting
Stub-O-Matic supports the ability to load stubs from various and multiple sources This means people can organise allthe stubs that support a particular service or use case together and any test that needs those stubs could pull them infrom common stub libraries maximising stub reuse
set foreign_url = httpwwwgooglecouk set other_url = httpwwwsuncom deletestubsscenario=smartbeginsessionscenario=smartampsession=smart_1ampmode=recordputstubsession=smart_1url=other_urlurl=foreign_urlputstubsession=smart_1text=random_rubbletext=response_textendsessionsession=smart_1
Command files are also programmable with Python code snippets See the example below
set session_name = smart_1 deletestubsscenario=smartbeginsessionscenario=smartampsession=session_nameampmode=recordputstubsession=session_namefirsttextMatcherfirstresponseendsessionsession=session_namebeginsessionscenario=smartampsession=session_nameampmode=playback make 3 requests for i in range(03) getresponsesession=session_namefirstrequest end endsessionsession=session_name
43 Passing Arguments
Arguments can be passed into command files and used in them The following example allows the session and scenarioto be named externally to the command file
execcmdscmdFile=staticcmdstestsacceptsmart_argcommandsampscen=bobampsession=smart_1
The above call can be used as follows
deletestubsscenario=scen[0]beginsessionscenario=scen[0]ampsession= session[0] ampmode=recordputstubsession=session[0]firsttextMatcherfirstresponseendsessionsession=session[0]beginsessionscenario=scen[0]ampsession=session[0]ampmode=playback for i in range(03) getresponsesession=session[0]firstrequest end endsessionsession=session[0]
12 Chapter 4 Stubo Command File (text format)
CHAPTER 5
GUI
51 Tracker
The Stubo lsquoTrackerrsquo page shows details of all requests submitted to Stub-O-Matic and the received response Clickon the link in the function column to show a detailed view Debug level tracking can be added to a getresponse callif lsquotracking_level=fullrsquo is added to the call Matching errors will also trigger some extra debug data to be added to anormal track record
Stubo can not keep tracker data forever so it will eventually drop off the server as space is used up
The tracker page shows all Stubo actions performed on the server Filters are provided to filter on session datetimeresponse times and errors
To go into detail on a particular transaction click the link in the lsquoFunctionrsquo column or lsquoDetailsrsquo in the lsquoStubo Responsersquocolumn For a getresponse call the matchers and responses for the call can be found from the gtgtgt view scenario details(stub matchers and responses) link at the top of the details page
Filtering The tracker filter fields should be self explanatory By default the tracker page shows only data sent to theStubo URL shown at the top of the browser To see tracker items from other URLs on the same hardware tick thelsquoAll Hostsrsquo box
52 Manage
The manage page allows users to execute command files and view system state
The list of loaded stubs includes the date the stubs were loaded into Stubo and the last time the stub was used Clickingthe scenario name shows the stored matchers and responses for the scenario
Also available is a command file scratchpad for entering one or more commands For example one could enter
endsessionsession=first_sessiondeletestubsscenario=firstendsessionsession=second_sessiondeletestubsscenario=second
All the above commands would then be executed See the results on the tracker page
The manage page also shows stubs delay policies and any external modules loaded
13
Stub-O-Matic Documentation Release 0X beta
53 Analytics
The lsquoAnalyticsrsquo tab takes you to a dashboard showing statistics about the Stubo server The orange bar along the topallows you to select different Stubo servers and time periods The first of the 4 squares on the right turns on a legend
Charts
chart title descriptionstubssecondresponse rate
number of stubs requested per second
delay added user defined delays to stubs these can be specified to emulate response times of thesystem being stubbed
latency average time taken to return a stub excluding network and user defined delaysrequestresponse size size in kb of the requests to stubo and responses deliveredhost response latency where Stubo is deployed across multiple servers this chart compares the different servers
against each otherfailure rate percent of failed requests (HTTP response gt= 400)health checks Stubo has a regular health check ping to demonstrate that all systems are working
properlyrunning putstubcount
cumulative total of stubs loaded
running responsecount
cumulative total of stubs retrieved
Also note the Stubo API documentation for getstats
14 Chapter 5 GUI
CHAPTER 6
Integration with Test Tools
Stub-O-Matic works with all major software test execution tools Stubo of course does not run tests or evaluateresults it emulates back-end systems needed by tests
Data dependent software tests typically consist of 3 steps
1 Data setup
2 Test execution
3 Data teardown
Working with Stubo to provide back-end data is no different
Stuborsquos API can be used to setup or teardown data directly Stubo command files offer a shorthand for using the APIand are much more compact Any test tool that can make HTTP calls can use Stubo
61 Test steps example
1 Trigger an http call similar to this from the test tool of your choice
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_setupcommands
2 Run tests
3 Trigger a teardown http call similar to this from the test tool
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_teardowncommands
Setup would typically i) delete stubs ii) set any delays iii) submit a recording via a series of putstub calls and finallyiv) begin a session in playback
Teardown would typically i) end session ii) delete stubs
Setup and teardown can be performed with a mixture of command files (YAML or commands) and direct HTTP APIcalls to stubo
15
Stub-O-Matic Documentation Release 0X beta
16 Chapter 6 Integration with Test Tools
CHAPTER 7
Stubbing
71 Add Delay to a Response
To emulate response times of systems being stubbed one can set a delay on stubs Delays are set on the stub responsenot on the request to stubo This is because one request into the system being tested can result in several back-endstubocalls each of which might need different delay settings
Stubo provides reusable delay_policy objects which can be used across many stubs and across sessions The de-lay_policy can be altered on the fly during a loadperformance test if desired There are currently two delay_policytypes to choose from (1) fixed and (2)random with mean and standard deviation
A fixed delay policy can be created or modified with a commnand such as
putdelay_policyname=a_fixed_delay_policyampdelay_type=fixedampmilliseconds=200
Note that the value for the lsquonamersquo can be set to any string and is used again in the putstub A variable delay can be setup as follows
putdelay_policyname=a_normalvariate_delay_policyampdelay_type=normalvariateampmean=100ampstddev=50
Allowed lsquodelay typersquo values are lsquofixedrsquo and lsquonormalvariatersquo The next step is to use the delay policy when loadingstubs eg
putstubsession=abcampdelay_policy=a_fixed_delay_policy
bull Whenever the stub above is used the named delay policy will be applied
bull Delay policies may be altered on the fly during a test run if desired To change a delay on the fly simply repeatthe putdelay_policy command with a different value Typically this would be done directly by the test toolrather than from a command file
bull Delay policies can be used across multiple sessions This allows for easier alteration of policies where manysessions use the same policy It also means that you should name policies carefully and only alter or sharepolicies you are in control of
72 Matching
Stub-O-Matic currently supports various types of matchers
17
Stub-O-Matic Documentation Release 0X beta
73 Body contains matching
One or more matchers can be defined whereby all matchers must be contained in the request to return a specifiedresponse A typical difficulty with system requests is that they often contain superfluous information such as sessionIDs or time stamps that get in the way of matching a request to the desired response One way Stub-O-Matic can solvethis problem is by the use of lsquomatchersrsquo Matchers are parts of the request that matter to you only what is needed tofind the correct response file
For example a request may include
ltsessionidgt123456ltsessionidgtlttransactionidgt0987654321-9999lttransactionidgt
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Of these 4 lines only the last 2 matter the first 2 lines will differ from one request to the next and be set by the systemmaking the request This request then will be different each time it is made How can one easily match a request likethis to a canned response file without writing code for each requestresponse pair The solution used by Stub-O-Maticis to not store requestresponse file pairs but rather store matchers and the response In the case above the matcherswould be
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Stub-O-Matic contains code which will take a request as input search for the response with the best matchers Everymatcher must be contained in the request to be a match When a match is found the corresponding response is servedback
All matchers must be present in the request to make a match Stub-O-Matic will return the response from the firstmatch if there is more than one possible match Note that whitespace and carriage returns are ignored when matching
Alternatives to removing parts of the request you donrsquot want to match on are
1 using a matcher template to substitute ids or dates etc from the request into the matcher OR
2 using a user exit to run custom code A convenience class XMLManglerExit is provided for XML payloads toignore XMl elements andor attributes from the matching process
74 Templated Matcher
This is an example of a template matcher than extracts values from the request into the matcher The xmltree object isan lxml parsed root element passed into the template
ltrequestgtlttransidgtxmltreexpath(requesttransid)[0]textlttransidgtltdtgt2009-10-24T000000+0000ltdtgtltflightnumbergt0455ltflightnumbergtltrequestgt
75 Templated Responses
A hand crafted Stubo response may contain variables which are substituted in For example a response may contain
set flight_nbr=XX1234 The flight number is flight_nbr
18 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
The resulting response served wil be
The flight number is XX1234
Code may be used for example adding random elements to a response
import random set rnd_nbr = randomrandint(1200) Hello World 1000 + rnd_nbr
The resulting response served will be
Hello World - followed by a number between 1000 and 1200
To put todayrsquos date in a response include the following
today_strformat(dmy)
Stubo responses are run through a Tornado template and any logic or commands allowed in these templates can beused See httpwwwtornadoweborgenstabletemplatehtml for details
76 Request Data in Responses
761 Text Manipulation
It is possible to pull data from the stub request and put it in the response Say you have 200 stubs that are identicalapart from a userid They could be condensed into one stub which uses the userid from the request in the responserequest
ltuseridgtabc123ltuseridgt
matcher
ltuseridgt
response template (using Python)
Hello to request_text[814]
response
Hello to abc123
762 XML Manipulation
Sometimes it is easier to use XPATH to pull data from a request and place it in a response If you need the SessionIdfrom this request
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtJohn1234ltwbsSessionIdgt
Pull the SessionId from the request and use it within the response template
76 Request Data in Responses 19
Stub-O-Matic Documentation Release 0X beta
set namespaces = soapenv httpschemasxmlsoaporgsoapenvelopewbs httpxmlaaacomws200901WBS_Session-20xsd
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtxmltreexpath(soapenvHeaderwbsSessionwbsSessionId
namespaces=namespaces)[0]textltwbsSessionIdgt
Note the xmltree variable is the parsed xml request (as an lxml Root object) made available to the template if therequest is valid xml
Another example - no namespaces in the request Request
ltxml version=10 encoding=UTF-8gtltCompensateCustomersCheck APCWCustom=-1 LocalCurrencyCode=GBP NumberOfAdults=1gt
Response
set currency_code = xmltreexpath(CompensateCustomersCheck)[0]attrib[LocalCurrencyCode] ltresponse LocalCurrencyCode=currency_codegtpay meltresponsegt
77 Stateful Stubs
State is important in simulating back-end systems For example if one was to check-in passenger Bob on a particularflight the back-end would allow this request to succeed only one time and subsequent attempts to check-in the samepassenger on the same flight would be rejected Stubo can emulate this functionality or if desired for repeatable testsStubo can allow Bob to be checked in multiple times on the same flight
Stateful stubbing is enabled automatically If stubs are recorded or loaded in which the same request returns differentresponses Stubo will remember this and internally keep track of which response should come next
If your test is to check-in Bob multiple times simply do not load the stubs representing the check-in rejection andStubo will allow multiple check-ins
78 User Exits
Stub-O-Matic provides hooks into the runtime to execute custom user code Stubo can execute your custom code totransform stub matchers requests andor responses This is a powerful mechanism when your stubs contain dynamicdata or need to change over time This can be useful to perform lsquointelligent stubbingrsquo for example to roll dates Theuser exit API is contained in the stuboextuser_exit module
781 API
782 XML Auto Mangling
If your data is XML a convenience class XMLUserExit is provided to enable auto mangling
20 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
783 API
79 Hooks
710 Modules
Modules are added with the putmodule command and referenced via the ext_module url arg in the putstub call Forexample
putmodulename=staticcmdstestsextxsltmanglerpydeletestubsscenario=mangler_xsltbeginsessionscenario=mangler_xsltampsession=mangler_xslt1ampmode=recordputstubsession=mangler_xslt1ampext_module=mangler firstrequest firstresponseendsessionsession=mangler_xslt1
This example dynamically imports a test mangler from the Stubo server Typically these external python modules thathook into the Stubo runtime would be sourced from source control system using a URL
Note user exit processing is only enabled if the lsquoext_modulersquo variable is set to a pre-loaded module during the putstubcall Each Stub-O-Matic host URL has its own copy of any loaded modules Such modules are available to any testand as such should typically be loaded once per test suite
711 Splitting
Useful to removing random elements from matchers It can happen that the system under test will include seeminglyrandom elements in the request This will cause the next run of the test to not match This problem can be solved byprocessing matchers when loading - during a record A matcher can be split into two with the variable text left outFuture request must match both of the matchers to get the response An example is
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N11111234567806P1ltCommandgtltCryptic_GetScreen_Querygt
Which becomes two matchers
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N1111
and
06P1ltCommandgtltCryptic_GetScreen_Querygt
cutting out the random number lsquo12345678rsquo in the middle
This matcher splitting can be done manually or automatically with a user exit Seestaticcmdstestsextsplitters1firstcommands for and example It uses the rules module at staticcmdstestsextaaapy
Multiple matchers excluding random request elements can be created manually and loaded as such
putstubsession=splitter_record splitter_1textMatcher splitter_2textMatcher splitterresponse
712 Caching Values
If emulating back-end behaviour means writing some code for a particularly tricky behaviour Stubo exposes a simplekey-value cache API to matchers and responses
79 Hooks 21
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
Stub-O-Matic Documentation Release 0X beta
Provide your requests hereplayback
scenario firstsession first_1requests-
json method GETbody timestamp 092345
get my stub
These will be listed on the Tracker page What just happened The command you made executed a Stub-O-Maticcommand file (demofirstyaml) That file loaded a stub into Stuborsquos database then simulated a request returning theexpected response These actions were logged and displayed on the Tracker page
There are a few key concepts that will make stubbing work
bull Scenario - A re-usable set of stubs
bull Session - The instance or use of the stubs from a scenario
bull Matcher - The matcher used to find the correct response for a request
bull Request - A message normally sent to a back-end by the system under test
bull Response - A message normally returned by a back-end system
bull Stub = Matcher(s) and a response
Stubo commands have been designed to be read written and used by either humans or the Stubo code The stub andcommand files are accessed via a URI (http) In practice this means putting them into a source code repositorysuch as Subversion (SVN) which you should be doing anyway The file location in the execcmds command can belocal to Stubo as in the firstyaml example or (more usual) a URL
For example you could run
stubodefaultexeccmdscmdfile=httpsyour-source-code-repostaticcmdsdemofirstyaml
22 Sessions and Scenarios
In order for stubs to be loaded and retrieved in a predictable automated manner by multiple users many Stubo eventsare controlled by a session A session is a unit of work that must be started and ended Sessions have two possiblemodes of operation
bull record - for loading stubs into the Stubo datastore
bull playback - for retrieving stubbed responses during testing
A scenario is a group of stubs that can be used and re-used by multiple sessions
23 Recording Stubs
Stubs can be created by hand or auto-recorded from a real system under test This is done from what we call a lsquoStubointegrator or Stubo client servicersquo A Stubo integrator is typically a custom library that intercepts real system calls andlsquore-playsrsquo them into a Stubo lsquorecordingrsquo via putstub commands Java and Python Stubo client libraries are provided
4 Chapter 2 Stub Basics
Stub-O-Matic Documentation Release 0X beta
to help integrate Stubo into your applications These clients are maintained in github at httpsgithubcomStub-O-Maticpython-client amp httpsgithubcomStub-O-Maticjava-client
If the real system changes its interface a recording can be repeated to capture the changes Of course this requires thatthe test back-end systems have the correct data in them to be used successfully
Stubo provides mechanisms to deal with context sensitive information such as transaction ids dates user names etcwhich makes recorded stubs less fragile
23 Recording Stubs 5
Stub-O-Matic Documentation Release 0X beta
6 Chapter 2 Stub Basics
CHAPTER 3
Stubo Command File
Command files are used to load stub files and add behaviour to them (state dates etc)
Stubs can also be loaded and tested using individual calls to the Stub-O-Matic server using the Stubo HTTP APICommand files are a shorthand for the API making it simpler to enable repeatable automated interactions with Stub-O-Matic by grouping a set of commands together required for a particular test
31 Command file (YAML)
An example
Stubo YAML Execution ordering is 1) commands 2) recording 3) playback
Commands go herecommands
-putmodule
varsname staticcmdstestsrestyamlnooppy
-putdelay_policy
varsname slowdelay_type fixedmilliseconds 1000
Describe your stubs hererecording
scenario restsession rest_recordingstubs-file stub1jsonvars
recorded_at as_date(2015-01-10)ext_module noop
-json
request method GETbodyPatterns [
7
Stub-O-Matic Documentation Release 0X beta
jsonpath [cmdx]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all ok 2headers Content-Type applicationjsonX-Custom-Header 1234
vars
foo bar
Provide your requests hereplayback
scenario restsession rest_playbackrequests-
file request1jsonvars
played_at as_date(2015-01-20)-
json method GETbody cmd x yheaders
Content-Type applicationjsonX-Custom-Header 1234
varsplayed_at as_date(2015-01-20)
Note the payloads can be defined either in place with the lsquojsonrsquo key or point to an external reference via the lsquofilersquo keyThe file can either reference a local statically served file as shown above or be a URI reference Typically it will be aURI reference to a location in a users source control system where their stubs are stored
The local file references in this example are shown below
stub1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
8 Chapter 3 Stubo Command File
Stub-O-Matic Documentation Release 0X beta
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
request1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
Note that these json payloads for the request and response are defined as strings Stubo also excepts the definition asdictionaries
32 Command Scripting
The YAML file is run through a template processor before being parsed and executed by Stubo Any variables definedsuch as lsquoplayed_atrsquo will evaluated and appropriate subsitutions made
A roll date example
(daterollyaml)
playbackrequests- file dateroll_1433930288_0requestresponse dateroll_1433930288_0stubo_responsevars
getresponse_arg this stub was played at 2015-06-10 095744839438
32 Command Scripting 9
Stub-O-Matic Documentation Release 0X beta
play_date 2014-09-12priority 1putstub_arg this stub was recorded at 2015-06-10 095744839387rec_date 2014-09-10tracking_level full
scenario daterollsession dateroll_1433930288
recordingscenario daterollsession dateroll_1433930288stubs- file dateroll_1433930288_0json
Referenced files
dateroll_1433930288_0json
priority 1args
priority 1rec_date 2014-09-10putstub_arg this stub was recorded at 2015-06-10 095744839387
request
bodyPatterns contains [
ltxml version=10 encoding=UTF-8 standalone=yes gtnltrollmegt n ltOriginDateTimegtroll_date(2014-09-10 as_date(rec_date) as_date(play_date))T000000ZltOriginDateTimegtnltrollmegt]
method POST
response
body ltresponsegtnltputstub_arggt raw putstub_arg ltputstub_arggtnltgetresponse_arggt getresponse_arg ltgetresponse_arggtnltresponsegtstatus 200
dateroll_1433930288_0request
body ltxml version=10 encoding=UTF-8 standalone=yesgtnltrollmegtn ltOriginDateTimegt2014-09-12T000000ZltOriginDateTimegtnltrollmegtheaders host nullpath nullquery uri nullmethod POST
10 Chapter 3 Stubo Command File
CHAPTER 4
Stubo Command File (text format)
The line orientated text version of the command file is an old format It does not support the matching required forREST For example you canrsquot match on the request headers or method It only supports one type of matcher whichis the lsquobody containsrsquo matcher If the text specified for the lsquobody containsrsquo matcher stub is found in the request amatch is found This is often however the only type of matcher required to successfully stub a system This format isstill supported for import via execcmds but will not be exported anymore If a text commands file is imported via alsquocommandsrsquo text file on export via getexport a lsquoyamlrsquo file will be exported
41 Text Command file
bull Lines beginning with a lsquorsquo are treated as comments
bull Commands follow the Stub-O-Matic REST API for example be-ginsessionscenario=abcampsession=12345ampmode=record
bull When the API requires something in the HTTP body input can be sourced by listing files after the command(separated by commas) Files are assumed to be in the same directory as the command file Alternatively a fullURI can be provided to source the input (matcher request or response)
bull Blank lines are ignored
bull An example command is putstubsession=12345rsquofeng_001textMatcher feng_002textMatcherfeng_001response A putstub command must have at least one matcher file and one response file Thisexample has two matchers
What goes in the command files
bull A textMatcher file could be some or all of the text from the request eg AB1234ltFlightInfoQuerygt
bull Multiple textMatchers files may be used in a stub All must evaluate to True against the request for a responseto be returned
bull request files contain the text of a request (eg text xml json) Note the using playback in commands is onlyused to test the stubs as these would be issued by the real system being tested in normal situations
bull response files contain the text of a response (eg text xml json)
bull Any naming convention can be used for matcher request and response files We have used suffixes oftextMatcher request amp response but you are free to use other extensions like matcherxml requestjson re-sponsexml etc
11
Stub-O-Matic Documentation Release 0X beta
42 Command Scripting
Stub-O-Matic supports the ability to load stubs from various and multiple sources This means people can organise allthe stubs that support a particular service or use case together and any test that needs those stubs could pull them infrom common stub libraries maximising stub reuse
set foreign_url = httpwwwgooglecouk set other_url = httpwwwsuncom deletestubsscenario=smartbeginsessionscenario=smartampsession=smart_1ampmode=recordputstubsession=smart_1url=other_urlurl=foreign_urlputstubsession=smart_1text=random_rubbletext=response_textendsessionsession=smart_1
Command files are also programmable with Python code snippets See the example below
set session_name = smart_1 deletestubsscenario=smartbeginsessionscenario=smartampsession=session_nameampmode=recordputstubsession=session_namefirsttextMatcherfirstresponseendsessionsession=session_namebeginsessionscenario=smartampsession=session_nameampmode=playback make 3 requests for i in range(03) getresponsesession=session_namefirstrequest end endsessionsession=session_name
43 Passing Arguments
Arguments can be passed into command files and used in them The following example allows the session and scenarioto be named externally to the command file
execcmdscmdFile=staticcmdstestsacceptsmart_argcommandsampscen=bobampsession=smart_1
The above call can be used as follows
deletestubsscenario=scen[0]beginsessionscenario=scen[0]ampsession= session[0] ampmode=recordputstubsession=session[0]firsttextMatcherfirstresponseendsessionsession=session[0]beginsessionscenario=scen[0]ampsession=session[0]ampmode=playback for i in range(03) getresponsesession=session[0]firstrequest end endsessionsession=session[0]
12 Chapter 4 Stubo Command File (text format)
CHAPTER 5
GUI
51 Tracker
The Stubo lsquoTrackerrsquo page shows details of all requests submitted to Stub-O-Matic and the received response Clickon the link in the function column to show a detailed view Debug level tracking can be added to a getresponse callif lsquotracking_level=fullrsquo is added to the call Matching errors will also trigger some extra debug data to be added to anormal track record
Stubo can not keep tracker data forever so it will eventually drop off the server as space is used up
The tracker page shows all Stubo actions performed on the server Filters are provided to filter on session datetimeresponse times and errors
To go into detail on a particular transaction click the link in the lsquoFunctionrsquo column or lsquoDetailsrsquo in the lsquoStubo Responsersquocolumn For a getresponse call the matchers and responses for the call can be found from the gtgtgt view scenario details(stub matchers and responses) link at the top of the details page
Filtering The tracker filter fields should be self explanatory By default the tracker page shows only data sent to theStubo URL shown at the top of the browser To see tracker items from other URLs on the same hardware tick thelsquoAll Hostsrsquo box
52 Manage
The manage page allows users to execute command files and view system state
The list of loaded stubs includes the date the stubs were loaded into Stubo and the last time the stub was used Clickingthe scenario name shows the stored matchers and responses for the scenario
Also available is a command file scratchpad for entering one or more commands For example one could enter
endsessionsession=first_sessiondeletestubsscenario=firstendsessionsession=second_sessiondeletestubsscenario=second
All the above commands would then be executed See the results on the tracker page
The manage page also shows stubs delay policies and any external modules loaded
13
Stub-O-Matic Documentation Release 0X beta
53 Analytics
The lsquoAnalyticsrsquo tab takes you to a dashboard showing statistics about the Stubo server The orange bar along the topallows you to select different Stubo servers and time periods The first of the 4 squares on the right turns on a legend
Charts
chart title descriptionstubssecondresponse rate
number of stubs requested per second
delay added user defined delays to stubs these can be specified to emulate response times of thesystem being stubbed
latency average time taken to return a stub excluding network and user defined delaysrequestresponse size size in kb of the requests to stubo and responses deliveredhost response latency where Stubo is deployed across multiple servers this chart compares the different servers
against each otherfailure rate percent of failed requests (HTTP response gt= 400)health checks Stubo has a regular health check ping to demonstrate that all systems are working
properlyrunning putstubcount
cumulative total of stubs loaded
running responsecount
cumulative total of stubs retrieved
Also note the Stubo API documentation for getstats
14 Chapter 5 GUI
CHAPTER 6
Integration with Test Tools
Stub-O-Matic works with all major software test execution tools Stubo of course does not run tests or evaluateresults it emulates back-end systems needed by tests
Data dependent software tests typically consist of 3 steps
1 Data setup
2 Test execution
3 Data teardown
Working with Stubo to provide back-end data is no different
Stuborsquos API can be used to setup or teardown data directly Stubo command files offer a shorthand for using the APIand are much more compact Any test tool that can make HTTP calls can use Stubo
61 Test steps example
1 Trigger an http call similar to this from the test tool of your choice
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_setupcommands
2 Run tests
3 Trigger a teardown http call similar to this from the test tool
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_teardowncommands
Setup would typically i) delete stubs ii) set any delays iii) submit a recording via a series of putstub calls and finallyiv) begin a session in playback
Teardown would typically i) end session ii) delete stubs
Setup and teardown can be performed with a mixture of command files (YAML or commands) and direct HTTP APIcalls to stubo
15
Stub-O-Matic Documentation Release 0X beta
16 Chapter 6 Integration with Test Tools
CHAPTER 7
Stubbing
71 Add Delay to a Response
To emulate response times of systems being stubbed one can set a delay on stubs Delays are set on the stub responsenot on the request to stubo This is because one request into the system being tested can result in several back-endstubocalls each of which might need different delay settings
Stubo provides reusable delay_policy objects which can be used across many stubs and across sessions The de-lay_policy can be altered on the fly during a loadperformance test if desired There are currently two delay_policytypes to choose from (1) fixed and (2)random with mean and standard deviation
A fixed delay policy can be created or modified with a commnand such as
putdelay_policyname=a_fixed_delay_policyampdelay_type=fixedampmilliseconds=200
Note that the value for the lsquonamersquo can be set to any string and is used again in the putstub A variable delay can be setup as follows
putdelay_policyname=a_normalvariate_delay_policyampdelay_type=normalvariateampmean=100ampstddev=50
Allowed lsquodelay typersquo values are lsquofixedrsquo and lsquonormalvariatersquo The next step is to use the delay policy when loadingstubs eg
putstubsession=abcampdelay_policy=a_fixed_delay_policy
bull Whenever the stub above is used the named delay policy will be applied
bull Delay policies may be altered on the fly during a test run if desired To change a delay on the fly simply repeatthe putdelay_policy command with a different value Typically this would be done directly by the test toolrather than from a command file
bull Delay policies can be used across multiple sessions This allows for easier alteration of policies where manysessions use the same policy It also means that you should name policies carefully and only alter or sharepolicies you are in control of
72 Matching
Stub-O-Matic currently supports various types of matchers
17
Stub-O-Matic Documentation Release 0X beta
73 Body contains matching
One or more matchers can be defined whereby all matchers must be contained in the request to return a specifiedresponse A typical difficulty with system requests is that they often contain superfluous information such as sessionIDs or time stamps that get in the way of matching a request to the desired response One way Stub-O-Matic can solvethis problem is by the use of lsquomatchersrsquo Matchers are parts of the request that matter to you only what is needed tofind the correct response file
For example a request may include
ltsessionidgt123456ltsessionidgtlttransactionidgt0987654321-9999lttransactionidgt
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Of these 4 lines only the last 2 matter the first 2 lines will differ from one request to the next and be set by the systemmaking the request This request then will be different each time it is made How can one easily match a request likethis to a canned response file without writing code for each requestresponse pair The solution used by Stub-O-Maticis to not store requestresponse file pairs but rather store matchers and the response In the case above the matcherswould be
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Stub-O-Matic contains code which will take a request as input search for the response with the best matchers Everymatcher must be contained in the request to be a match When a match is found the corresponding response is servedback
All matchers must be present in the request to make a match Stub-O-Matic will return the response from the firstmatch if there is more than one possible match Note that whitespace and carriage returns are ignored when matching
Alternatives to removing parts of the request you donrsquot want to match on are
1 using a matcher template to substitute ids or dates etc from the request into the matcher OR
2 using a user exit to run custom code A convenience class XMLManglerExit is provided for XML payloads toignore XMl elements andor attributes from the matching process
74 Templated Matcher
This is an example of a template matcher than extracts values from the request into the matcher The xmltree object isan lxml parsed root element passed into the template
ltrequestgtlttransidgtxmltreexpath(requesttransid)[0]textlttransidgtltdtgt2009-10-24T000000+0000ltdtgtltflightnumbergt0455ltflightnumbergtltrequestgt
75 Templated Responses
A hand crafted Stubo response may contain variables which are substituted in For example a response may contain
set flight_nbr=XX1234 The flight number is flight_nbr
18 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
The resulting response served wil be
The flight number is XX1234
Code may be used for example adding random elements to a response
import random set rnd_nbr = randomrandint(1200) Hello World 1000 + rnd_nbr
The resulting response served will be
Hello World - followed by a number between 1000 and 1200
To put todayrsquos date in a response include the following
today_strformat(dmy)
Stubo responses are run through a Tornado template and any logic or commands allowed in these templates can beused See httpwwwtornadoweborgenstabletemplatehtml for details
76 Request Data in Responses
761 Text Manipulation
It is possible to pull data from the stub request and put it in the response Say you have 200 stubs that are identicalapart from a userid They could be condensed into one stub which uses the userid from the request in the responserequest
ltuseridgtabc123ltuseridgt
matcher
ltuseridgt
response template (using Python)
Hello to request_text[814]
response
Hello to abc123
762 XML Manipulation
Sometimes it is easier to use XPATH to pull data from a request and place it in a response If you need the SessionIdfrom this request
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtJohn1234ltwbsSessionIdgt
Pull the SessionId from the request and use it within the response template
76 Request Data in Responses 19
Stub-O-Matic Documentation Release 0X beta
set namespaces = soapenv httpschemasxmlsoaporgsoapenvelopewbs httpxmlaaacomws200901WBS_Session-20xsd
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtxmltreexpath(soapenvHeaderwbsSessionwbsSessionId
namespaces=namespaces)[0]textltwbsSessionIdgt
Note the xmltree variable is the parsed xml request (as an lxml Root object) made available to the template if therequest is valid xml
Another example - no namespaces in the request Request
ltxml version=10 encoding=UTF-8gtltCompensateCustomersCheck APCWCustom=-1 LocalCurrencyCode=GBP NumberOfAdults=1gt
Response
set currency_code = xmltreexpath(CompensateCustomersCheck)[0]attrib[LocalCurrencyCode] ltresponse LocalCurrencyCode=currency_codegtpay meltresponsegt
77 Stateful Stubs
State is important in simulating back-end systems For example if one was to check-in passenger Bob on a particularflight the back-end would allow this request to succeed only one time and subsequent attempts to check-in the samepassenger on the same flight would be rejected Stubo can emulate this functionality or if desired for repeatable testsStubo can allow Bob to be checked in multiple times on the same flight
Stateful stubbing is enabled automatically If stubs are recorded or loaded in which the same request returns differentresponses Stubo will remember this and internally keep track of which response should come next
If your test is to check-in Bob multiple times simply do not load the stubs representing the check-in rejection andStubo will allow multiple check-ins
78 User Exits
Stub-O-Matic provides hooks into the runtime to execute custom user code Stubo can execute your custom code totransform stub matchers requests andor responses This is a powerful mechanism when your stubs contain dynamicdata or need to change over time This can be useful to perform lsquointelligent stubbingrsquo for example to roll dates Theuser exit API is contained in the stuboextuser_exit module
781 API
782 XML Auto Mangling
If your data is XML a convenience class XMLUserExit is provided to enable auto mangling
20 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
783 API
79 Hooks
710 Modules
Modules are added with the putmodule command and referenced via the ext_module url arg in the putstub call Forexample
putmodulename=staticcmdstestsextxsltmanglerpydeletestubsscenario=mangler_xsltbeginsessionscenario=mangler_xsltampsession=mangler_xslt1ampmode=recordputstubsession=mangler_xslt1ampext_module=mangler firstrequest firstresponseendsessionsession=mangler_xslt1
This example dynamically imports a test mangler from the Stubo server Typically these external python modules thathook into the Stubo runtime would be sourced from source control system using a URL
Note user exit processing is only enabled if the lsquoext_modulersquo variable is set to a pre-loaded module during the putstubcall Each Stub-O-Matic host URL has its own copy of any loaded modules Such modules are available to any testand as such should typically be loaded once per test suite
711 Splitting
Useful to removing random elements from matchers It can happen that the system under test will include seeminglyrandom elements in the request This will cause the next run of the test to not match This problem can be solved byprocessing matchers when loading - during a record A matcher can be split into two with the variable text left outFuture request must match both of the matchers to get the response An example is
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N11111234567806P1ltCommandgtltCryptic_GetScreen_Querygt
Which becomes two matchers
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N1111
and
06P1ltCommandgtltCryptic_GetScreen_Querygt
cutting out the random number lsquo12345678rsquo in the middle
This matcher splitting can be done manually or automatically with a user exit Seestaticcmdstestsextsplitters1firstcommands for and example It uses the rules module at staticcmdstestsextaaapy
Multiple matchers excluding random request elements can be created manually and loaded as such
putstubsession=splitter_record splitter_1textMatcher splitter_2textMatcher splitterresponse
712 Caching Values
If emulating back-end behaviour means writing some code for a particularly tricky behaviour Stubo exposes a simplekey-value cache API to matchers and responses
79 Hooks 21
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
Stub-O-Matic Documentation Release 0X beta
to help integrate Stubo into your applications These clients are maintained in github at httpsgithubcomStub-O-Maticpython-client amp httpsgithubcomStub-O-Maticjava-client
If the real system changes its interface a recording can be repeated to capture the changes Of course this requires thatthe test back-end systems have the correct data in them to be used successfully
Stubo provides mechanisms to deal with context sensitive information such as transaction ids dates user names etcwhich makes recorded stubs less fragile
23 Recording Stubs 5
Stub-O-Matic Documentation Release 0X beta
6 Chapter 2 Stub Basics
CHAPTER 3
Stubo Command File
Command files are used to load stub files and add behaviour to them (state dates etc)
Stubs can also be loaded and tested using individual calls to the Stub-O-Matic server using the Stubo HTTP APICommand files are a shorthand for the API making it simpler to enable repeatable automated interactions with Stub-O-Matic by grouping a set of commands together required for a particular test
31 Command file (YAML)
An example
Stubo YAML Execution ordering is 1) commands 2) recording 3) playback
Commands go herecommands
-putmodule
varsname staticcmdstestsrestyamlnooppy
-putdelay_policy
varsname slowdelay_type fixedmilliseconds 1000
Describe your stubs hererecording
scenario restsession rest_recordingstubs-file stub1jsonvars
recorded_at as_date(2015-01-10)ext_module noop
-json
request method GETbodyPatterns [
7
Stub-O-Matic Documentation Release 0X beta
jsonpath [cmdx]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all ok 2headers Content-Type applicationjsonX-Custom-Header 1234
vars
foo bar
Provide your requests hereplayback
scenario restsession rest_playbackrequests-
file request1jsonvars
played_at as_date(2015-01-20)-
json method GETbody cmd x yheaders
Content-Type applicationjsonX-Custom-Header 1234
varsplayed_at as_date(2015-01-20)
Note the payloads can be defined either in place with the lsquojsonrsquo key or point to an external reference via the lsquofilersquo keyThe file can either reference a local statically served file as shown above or be a URI reference Typically it will be aURI reference to a location in a users source control system where their stubs are stored
The local file references in this example are shown below
stub1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
8 Chapter 3 Stubo Command File
Stub-O-Matic Documentation Release 0X beta
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
request1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
Note that these json payloads for the request and response are defined as strings Stubo also excepts the definition asdictionaries
32 Command Scripting
The YAML file is run through a template processor before being parsed and executed by Stubo Any variables definedsuch as lsquoplayed_atrsquo will evaluated and appropriate subsitutions made
A roll date example
(daterollyaml)
playbackrequests- file dateroll_1433930288_0requestresponse dateroll_1433930288_0stubo_responsevars
getresponse_arg this stub was played at 2015-06-10 095744839438
32 Command Scripting 9
Stub-O-Matic Documentation Release 0X beta
play_date 2014-09-12priority 1putstub_arg this stub was recorded at 2015-06-10 095744839387rec_date 2014-09-10tracking_level full
scenario daterollsession dateroll_1433930288
recordingscenario daterollsession dateroll_1433930288stubs- file dateroll_1433930288_0json
Referenced files
dateroll_1433930288_0json
priority 1args
priority 1rec_date 2014-09-10putstub_arg this stub was recorded at 2015-06-10 095744839387
request
bodyPatterns contains [
ltxml version=10 encoding=UTF-8 standalone=yes gtnltrollmegt n ltOriginDateTimegtroll_date(2014-09-10 as_date(rec_date) as_date(play_date))T000000ZltOriginDateTimegtnltrollmegt]
method POST
response
body ltresponsegtnltputstub_arggt raw putstub_arg ltputstub_arggtnltgetresponse_arggt getresponse_arg ltgetresponse_arggtnltresponsegtstatus 200
dateroll_1433930288_0request
body ltxml version=10 encoding=UTF-8 standalone=yesgtnltrollmegtn ltOriginDateTimegt2014-09-12T000000ZltOriginDateTimegtnltrollmegtheaders host nullpath nullquery uri nullmethod POST
10 Chapter 3 Stubo Command File
CHAPTER 4
Stubo Command File (text format)
The line orientated text version of the command file is an old format It does not support the matching required forREST For example you canrsquot match on the request headers or method It only supports one type of matcher whichis the lsquobody containsrsquo matcher If the text specified for the lsquobody containsrsquo matcher stub is found in the request amatch is found This is often however the only type of matcher required to successfully stub a system This format isstill supported for import via execcmds but will not be exported anymore If a text commands file is imported via alsquocommandsrsquo text file on export via getexport a lsquoyamlrsquo file will be exported
41 Text Command file
bull Lines beginning with a lsquorsquo are treated as comments
bull Commands follow the Stub-O-Matic REST API for example be-ginsessionscenario=abcampsession=12345ampmode=record
bull When the API requires something in the HTTP body input can be sourced by listing files after the command(separated by commas) Files are assumed to be in the same directory as the command file Alternatively a fullURI can be provided to source the input (matcher request or response)
bull Blank lines are ignored
bull An example command is putstubsession=12345rsquofeng_001textMatcher feng_002textMatcherfeng_001response A putstub command must have at least one matcher file and one response file Thisexample has two matchers
What goes in the command files
bull A textMatcher file could be some or all of the text from the request eg AB1234ltFlightInfoQuerygt
bull Multiple textMatchers files may be used in a stub All must evaluate to True against the request for a responseto be returned
bull request files contain the text of a request (eg text xml json) Note the using playback in commands is onlyused to test the stubs as these would be issued by the real system being tested in normal situations
bull response files contain the text of a response (eg text xml json)
bull Any naming convention can be used for matcher request and response files We have used suffixes oftextMatcher request amp response but you are free to use other extensions like matcherxml requestjson re-sponsexml etc
11
Stub-O-Matic Documentation Release 0X beta
42 Command Scripting
Stub-O-Matic supports the ability to load stubs from various and multiple sources This means people can organise allthe stubs that support a particular service or use case together and any test that needs those stubs could pull them infrom common stub libraries maximising stub reuse
set foreign_url = httpwwwgooglecouk set other_url = httpwwwsuncom deletestubsscenario=smartbeginsessionscenario=smartampsession=smart_1ampmode=recordputstubsession=smart_1url=other_urlurl=foreign_urlputstubsession=smart_1text=random_rubbletext=response_textendsessionsession=smart_1
Command files are also programmable with Python code snippets See the example below
set session_name = smart_1 deletestubsscenario=smartbeginsessionscenario=smartampsession=session_nameampmode=recordputstubsession=session_namefirsttextMatcherfirstresponseendsessionsession=session_namebeginsessionscenario=smartampsession=session_nameampmode=playback make 3 requests for i in range(03) getresponsesession=session_namefirstrequest end endsessionsession=session_name
43 Passing Arguments
Arguments can be passed into command files and used in them The following example allows the session and scenarioto be named externally to the command file
execcmdscmdFile=staticcmdstestsacceptsmart_argcommandsampscen=bobampsession=smart_1
The above call can be used as follows
deletestubsscenario=scen[0]beginsessionscenario=scen[0]ampsession= session[0] ampmode=recordputstubsession=session[0]firsttextMatcherfirstresponseendsessionsession=session[0]beginsessionscenario=scen[0]ampsession=session[0]ampmode=playback for i in range(03) getresponsesession=session[0]firstrequest end endsessionsession=session[0]
12 Chapter 4 Stubo Command File (text format)
CHAPTER 5
GUI
51 Tracker
The Stubo lsquoTrackerrsquo page shows details of all requests submitted to Stub-O-Matic and the received response Clickon the link in the function column to show a detailed view Debug level tracking can be added to a getresponse callif lsquotracking_level=fullrsquo is added to the call Matching errors will also trigger some extra debug data to be added to anormal track record
Stubo can not keep tracker data forever so it will eventually drop off the server as space is used up
The tracker page shows all Stubo actions performed on the server Filters are provided to filter on session datetimeresponse times and errors
To go into detail on a particular transaction click the link in the lsquoFunctionrsquo column or lsquoDetailsrsquo in the lsquoStubo Responsersquocolumn For a getresponse call the matchers and responses for the call can be found from the gtgtgt view scenario details(stub matchers and responses) link at the top of the details page
Filtering The tracker filter fields should be self explanatory By default the tracker page shows only data sent to theStubo URL shown at the top of the browser To see tracker items from other URLs on the same hardware tick thelsquoAll Hostsrsquo box
52 Manage
The manage page allows users to execute command files and view system state
The list of loaded stubs includes the date the stubs were loaded into Stubo and the last time the stub was used Clickingthe scenario name shows the stored matchers and responses for the scenario
Also available is a command file scratchpad for entering one or more commands For example one could enter
endsessionsession=first_sessiondeletestubsscenario=firstendsessionsession=second_sessiondeletestubsscenario=second
All the above commands would then be executed See the results on the tracker page
The manage page also shows stubs delay policies and any external modules loaded
13
Stub-O-Matic Documentation Release 0X beta
53 Analytics
The lsquoAnalyticsrsquo tab takes you to a dashboard showing statistics about the Stubo server The orange bar along the topallows you to select different Stubo servers and time periods The first of the 4 squares on the right turns on a legend
Charts
chart title descriptionstubssecondresponse rate
number of stubs requested per second
delay added user defined delays to stubs these can be specified to emulate response times of thesystem being stubbed
latency average time taken to return a stub excluding network and user defined delaysrequestresponse size size in kb of the requests to stubo and responses deliveredhost response latency where Stubo is deployed across multiple servers this chart compares the different servers
against each otherfailure rate percent of failed requests (HTTP response gt= 400)health checks Stubo has a regular health check ping to demonstrate that all systems are working
properlyrunning putstubcount
cumulative total of stubs loaded
running responsecount
cumulative total of stubs retrieved
Also note the Stubo API documentation for getstats
14 Chapter 5 GUI
CHAPTER 6
Integration with Test Tools
Stub-O-Matic works with all major software test execution tools Stubo of course does not run tests or evaluateresults it emulates back-end systems needed by tests
Data dependent software tests typically consist of 3 steps
1 Data setup
2 Test execution
3 Data teardown
Working with Stubo to provide back-end data is no different
Stuborsquos API can be used to setup or teardown data directly Stubo command files offer a shorthand for using the APIand are much more compact Any test tool that can make HTTP calls can use Stubo
61 Test steps example
1 Trigger an http call similar to this from the test tool of your choice
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_setupcommands
2 Run tests
3 Trigger a teardown http call similar to this from the test tool
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_teardowncommands
Setup would typically i) delete stubs ii) set any delays iii) submit a recording via a series of putstub calls and finallyiv) begin a session in playback
Teardown would typically i) end session ii) delete stubs
Setup and teardown can be performed with a mixture of command files (YAML or commands) and direct HTTP APIcalls to stubo
15
Stub-O-Matic Documentation Release 0X beta
16 Chapter 6 Integration with Test Tools
CHAPTER 7
Stubbing
71 Add Delay to a Response
To emulate response times of systems being stubbed one can set a delay on stubs Delays are set on the stub responsenot on the request to stubo This is because one request into the system being tested can result in several back-endstubocalls each of which might need different delay settings
Stubo provides reusable delay_policy objects which can be used across many stubs and across sessions The de-lay_policy can be altered on the fly during a loadperformance test if desired There are currently two delay_policytypes to choose from (1) fixed and (2)random with mean and standard deviation
A fixed delay policy can be created or modified with a commnand such as
putdelay_policyname=a_fixed_delay_policyampdelay_type=fixedampmilliseconds=200
Note that the value for the lsquonamersquo can be set to any string and is used again in the putstub A variable delay can be setup as follows
putdelay_policyname=a_normalvariate_delay_policyampdelay_type=normalvariateampmean=100ampstddev=50
Allowed lsquodelay typersquo values are lsquofixedrsquo and lsquonormalvariatersquo The next step is to use the delay policy when loadingstubs eg
putstubsession=abcampdelay_policy=a_fixed_delay_policy
bull Whenever the stub above is used the named delay policy will be applied
bull Delay policies may be altered on the fly during a test run if desired To change a delay on the fly simply repeatthe putdelay_policy command with a different value Typically this would be done directly by the test toolrather than from a command file
bull Delay policies can be used across multiple sessions This allows for easier alteration of policies where manysessions use the same policy It also means that you should name policies carefully and only alter or sharepolicies you are in control of
72 Matching
Stub-O-Matic currently supports various types of matchers
17
Stub-O-Matic Documentation Release 0X beta
73 Body contains matching
One or more matchers can be defined whereby all matchers must be contained in the request to return a specifiedresponse A typical difficulty with system requests is that they often contain superfluous information such as sessionIDs or time stamps that get in the way of matching a request to the desired response One way Stub-O-Matic can solvethis problem is by the use of lsquomatchersrsquo Matchers are parts of the request that matter to you only what is needed tofind the correct response file
For example a request may include
ltsessionidgt123456ltsessionidgtlttransactionidgt0987654321-9999lttransactionidgt
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Of these 4 lines only the last 2 matter the first 2 lines will differ from one request to the next and be set by the systemmaking the request This request then will be different each time it is made How can one easily match a request likethis to a canned response file without writing code for each requestresponse pair The solution used by Stub-O-Maticis to not store requestresponse file pairs but rather store matchers and the response In the case above the matcherswould be
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Stub-O-Matic contains code which will take a request as input search for the response with the best matchers Everymatcher must be contained in the request to be a match When a match is found the corresponding response is servedback
All matchers must be present in the request to make a match Stub-O-Matic will return the response from the firstmatch if there is more than one possible match Note that whitespace and carriage returns are ignored when matching
Alternatives to removing parts of the request you donrsquot want to match on are
1 using a matcher template to substitute ids or dates etc from the request into the matcher OR
2 using a user exit to run custom code A convenience class XMLManglerExit is provided for XML payloads toignore XMl elements andor attributes from the matching process
74 Templated Matcher
This is an example of a template matcher than extracts values from the request into the matcher The xmltree object isan lxml parsed root element passed into the template
ltrequestgtlttransidgtxmltreexpath(requesttransid)[0]textlttransidgtltdtgt2009-10-24T000000+0000ltdtgtltflightnumbergt0455ltflightnumbergtltrequestgt
75 Templated Responses
A hand crafted Stubo response may contain variables which are substituted in For example a response may contain
set flight_nbr=XX1234 The flight number is flight_nbr
18 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
The resulting response served wil be
The flight number is XX1234
Code may be used for example adding random elements to a response
import random set rnd_nbr = randomrandint(1200) Hello World 1000 + rnd_nbr
The resulting response served will be
Hello World - followed by a number between 1000 and 1200
To put todayrsquos date in a response include the following
today_strformat(dmy)
Stubo responses are run through a Tornado template and any logic or commands allowed in these templates can beused See httpwwwtornadoweborgenstabletemplatehtml for details
76 Request Data in Responses
761 Text Manipulation
It is possible to pull data from the stub request and put it in the response Say you have 200 stubs that are identicalapart from a userid They could be condensed into one stub which uses the userid from the request in the responserequest
ltuseridgtabc123ltuseridgt
matcher
ltuseridgt
response template (using Python)
Hello to request_text[814]
response
Hello to abc123
762 XML Manipulation
Sometimes it is easier to use XPATH to pull data from a request and place it in a response If you need the SessionIdfrom this request
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtJohn1234ltwbsSessionIdgt
Pull the SessionId from the request and use it within the response template
76 Request Data in Responses 19
Stub-O-Matic Documentation Release 0X beta
set namespaces = soapenv httpschemasxmlsoaporgsoapenvelopewbs httpxmlaaacomws200901WBS_Session-20xsd
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtxmltreexpath(soapenvHeaderwbsSessionwbsSessionId
namespaces=namespaces)[0]textltwbsSessionIdgt
Note the xmltree variable is the parsed xml request (as an lxml Root object) made available to the template if therequest is valid xml
Another example - no namespaces in the request Request
ltxml version=10 encoding=UTF-8gtltCompensateCustomersCheck APCWCustom=-1 LocalCurrencyCode=GBP NumberOfAdults=1gt
Response
set currency_code = xmltreexpath(CompensateCustomersCheck)[0]attrib[LocalCurrencyCode] ltresponse LocalCurrencyCode=currency_codegtpay meltresponsegt
77 Stateful Stubs
State is important in simulating back-end systems For example if one was to check-in passenger Bob on a particularflight the back-end would allow this request to succeed only one time and subsequent attempts to check-in the samepassenger on the same flight would be rejected Stubo can emulate this functionality or if desired for repeatable testsStubo can allow Bob to be checked in multiple times on the same flight
Stateful stubbing is enabled automatically If stubs are recorded or loaded in which the same request returns differentresponses Stubo will remember this and internally keep track of which response should come next
If your test is to check-in Bob multiple times simply do not load the stubs representing the check-in rejection andStubo will allow multiple check-ins
78 User Exits
Stub-O-Matic provides hooks into the runtime to execute custom user code Stubo can execute your custom code totransform stub matchers requests andor responses This is a powerful mechanism when your stubs contain dynamicdata or need to change over time This can be useful to perform lsquointelligent stubbingrsquo for example to roll dates Theuser exit API is contained in the stuboextuser_exit module
781 API
782 XML Auto Mangling
If your data is XML a convenience class XMLUserExit is provided to enable auto mangling
20 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
783 API
79 Hooks
710 Modules
Modules are added with the putmodule command and referenced via the ext_module url arg in the putstub call Forexample
putmodulename=staticcmdstestsextxsltmanglerpydeletestubsscenario=mangler_xsltbeginsessionscenario=mangler_xsltampsession=mangler_xslt1ampmode=recordputstubsession=mangler_xslt1ampext_module=mangler firstrequest firstresponseendsessionsession=mangler_xslt1
This example dynamically imports a test mangler from the Stubo server Typically these external python modules thathook into the Stubo runtime would be sourced from source control system using a URL
Note user exit processing is only enabled if the lsquoext_modulersquo variable is set to a pre-loaded module during the putstubcall Each Stub-O-Matic host URL has its own copy of any loaded modules Such modules are available to any testand as such should typically be loaded once per test suite
711 Splitting
Useful to removing random elements from matchers It can happen that the system under test will include seeminglyrandom elements in the request This will cause the next run of the test to not match This problem can be solved byprocessing matchers when loading - during a record A matcher can be split into two with the variable text left outFuture request must match both of the matchers to get the response An example is
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N11111234567806P1ltCommandgtltCryptic_GetScreen_Querygt
Which becomes two matchers
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N1111
and
06P1ltCommandgtltCryptic_GetScreen_Querygt
cutting out the random number lsquo12345678rsquo in the middle
This matcher splitting can be done manually or automatically with a user exit Seestaticcmdstestsextsplitters1firstcommands for and example It uses the rules module at staticcmdstestsextaaapy
Multiple matchers excluding random request elements can be created manually and loaded as such
putstubsession=splitter_record splitter_1textMatcher splitter_2textMatcher splitterresponse
712 Caching Values
If emulating back-end behaviour means writing some code for a particularly tricky behaviour Stubo exposes a simplekey-value cache API to matchers and responses
79 Hooks 21
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
Stub-O-Matic Documentation Release 0X beta
6 Chapter 2 Stub Basics
CHAPTER 3
Stubo Command File
Command files are used to load stub files and add behaviour to them (state dates etc)
Stubs can also be loaded and tested using individual calls to the Stub-O-Matic server using the Stubo HTTP APICommand files are a shorthand for the API making it simpler to enable repeatable automated interactions with Stub-O-Matic by grouping a set of commands together required for a particular test
31 Command file (YAML)
An example
Stubo YAML Execution ordering is 1) commands 2) recording 3) playback
Commands go herecommands
-putmodule
varsname staticcmdstestsrestyamlnooppy
-putdelay_policy
varsname slowdelay_type fixedmilliseconds 1000
Describe your stubs hererecording
scenario restsession rest_recordingstubs-file stub1jsonvars
recorded_at as_date(2015-01-10)ext_module noop
-json
request method GETbodyPatterns [
7
Stub-O-Matic Documentation Release 0X beta
jsonpath [cmdx]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all ok 2headers Content-Type applicationjsonX-Custom-Header 1234
vars
foo bar
Provide your requests hereplayback
scenario restsession rest_playbackrequests-
file request1jsonvars
played_at as_date(2015-01-20)-
json method GETbody cmd x yheaders
Content-Type applicationjsonX-Custom-Header 1234
varsplayed_at as_date(2015-01-20)
Note the payloads can be defined either in place with the lsquojsonrsquo key or point to an external reference via the lsquofilersquo keyThe file can either reference a local statically served file as shown above or be a URI reference Typically it will be aURI reference to a location in a users source control system where their stubs are stored
The local file references in this example are shown below
stub1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
8 Chapter 3 Stubo Command File
Stub-O-Matic Documentation Release 0X beta
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
request1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
Note that these json payloads for the request and response are defined as strings Stubo also excepts the definition asdictionaries
32 Command Scripting
The YAML file is run through a template processor before being parsed and executed by Stubo Any variables definedsuch as lsquoplayed_atrsquo will evaluated and appropriate subsitutions made
A roll date example
(daterollyaml)
playbackrequests- file dateroll_1433930288_0requestresponse dateroll_1433930288_0stubo_responsevars
getresponse_arg this stub was played at 2015-06-10 095744839438
32 Command Scripting 9
Stub-O-Matic Documentation Release 0X beta
play_date 2014-09-12priority 1putstub_arg this stub was recorded at 2015-06-10 095744839387rec_date 2014-09-10tracking_level full
scenario daterollsession dateroll_1433930288
recordingscenario daterollsession dateroll_1433930288stubs- file dateroll_1433930288_0json
Referenced files
dateroll_1433930288_0json
priority 1args
priority 1rec_date 2014-09-10putstub_arg this stub was recorded at 2015-06-10 095744839387
request
bodyPatterns contains [
ltxml version=10 encoding=UTF-8 standalone=yes gtnltrollmegt n ltOriginDateTimegtroll_date(2014-09-10 as_date(rec_date) as_date(play_date))T000000ZltOriginDateTimegtnltrollmegt]
method POST
response
body ltresponsegtnltputstub_arggt raw putstub_arg ltputstub_arggtnltgetresponse_arggt getresponse_arg ltgetresponse_arggtnltresponsegtstatus 200
dateroll_1433930288_0request
body ltxml version=10 encoding=UTF-8 standalone=yesgtnltrollmegtn ltOriginDateTimegt2014-09-12T000000ZltOriginDateTimegtnltrollmegtheaders host nullpath nullquery uri nullmethod POST
10 Chapter 3 Stubo Command File
CHAPTER 4
Stubo Command File (text format)
The line orientated text version of the command file is an old format It does not support the matching required forREST For example you canrsquot match on the request headers or method It only supports one type of matcher whichis the lsquobody containsrsquo matcher If the text specified for the lsquobody containsrsquo matcher stub is found in the request amatch is found This is often however the only type of matcher required to successfully stub a system This format isstill supported for import via execcmds but will not be exported anymore If a text commands file is imported via alsquocommandsrsquo text file on export via getexport a lsquoyamlrsquo file will be exported
41 Text Command file
bull Lines beginning with a lsquorsquo are treated as comments
bull Commands follow the Stub-O-Matic REST API for example be-ginsessionscenario=abcampsession=12345ampmode=record
bull When the API requires something in the HTTP body input can be sourced by listing files after the command(separated by commas) Files are assumed to be in the same directory as the command file Alternatively a fullURI can be provided to source the input (matcher request or response)
bull Blank lines are ignored
bull An example command is putstubsession=12345rsquofeng_001textMatcher feng_002textMatcherfeng_001response A putstub command must have at least one matcher file and one response file Thisexample has two matchers
What goes in the command files
bull A textMatcher file could be some or all of the text from the request eg AB1234ltFlightInfoQuerygt
bull Multiple textMatchers files may be used in a stub All must evaluate to True against the request for a responseto be returned
bull request files contain the text of a request (eg text xml json) Note the using playback in commands is onlyused to test the stubs as these would be issued by the real system being tested in normal situations
bull response files contain the text of a response (eg text xml json)
bull Any naming convention can be used for matcher request and response files We have used suffixes oftextMatcher request amp response but you are free to use other extensions like matcherxml requestjson re-sponsexml etc
11
Stub-O-Matic Documentation Release 0X beta
42 Command Scripting
Stub-O-Matic supports the ability to load stubs from various and multiple sources This means people can organise allthe stubs that support a particular service or use case together and any test that needs those stubs could pull them infrom common stub libraries maximising stub reuse
set foreign_url = httpwwwgooglecouk set other_url = httpwwwsuncom deletestubsscenario=smartbeginsessionscenario=smartampsession=smart_1ampmode=recordputstubsession=smart_1url=other_urlurl=foreign_urlputstubsession=smart_1text=random_rubbletext=response_textendsessionsession=smart_1
Command files are also programmable with Python code snippets See the example below
set session_name = smart_1 deletestubsscenario=smartbeginsessionscenario=smartampsession=session_nameampmode=recordputstubsession=session_namefirsttextMatcherfirstresponseendsessionsession=session_namebeginsessionscenario=smartampsession=session_nameampmode=playback make 3 requests for i in range(03) getresponsesession=session_namefirstrequest end endsessionsession=session_name
43 Passing Arguments
Arguments can be passed into command files and used in them The following example allows the session and scenarioto be named externally to the command file
execcmdscmdFile=staticcmdstestsacceptsmart_argcommandsampscen=bobampsession=smart_1
The above call can be used as follows
deletestubsscenario=scen[0]beginsessionscenario=scen[0]ampsession= session[0] ampmode=recordputstubsession=session[0]firsttextMatcherfirstresponseendsessionsession=session[0]beginsessionscenario=scen[0]ampsession=session[0]ampmode=playback for i in range(03) getresponsesession=session[0]firstrequest end endsessionsession=session[0]
12 Chapter 4 Stubo Command File (text format)
CHAPTER 5
GUI
51 Tracker
The Stubo lsquoTrackerrsquo page shows details of all requests submitted to Stub-O-Matic and the received response Clickon the link in the function column to show a detailed view Debug level tracking can be added to a getresponse callif lsquotracking_level=fullrsquo is added to the call Matching errors will also trigger some extra debug data to be added to anormal track record
Stubo can not keep tracker data forever so it will eventually drop off the server as space is used up
The tracker page shows all Stubo actions performed on the server Filters are provided to filter on session datetimeresponse times and errors
To go into detail on a particular transaction click the link in the lsquoFunctionrsquo column or lsquoDetailsrsquo in the lsquoStubo Responsersquocolumn For a getresponse call the matchers and responses for the call can be found from the gtgtgt view scenario details(stub matchers and responses) link at the top of the details page
Filtering The tracker filter fields should be self explanatory By default the tracker page shows only data sent to theStubo URL shown at the top of the browser To see tracker items from other URLs on the same hardware tick thelsquoAll Hostsrsquo box
52 Manage
The manage page allows users to execute command files and view system state
The list of loaded stubs includes the date the stubs were loaded into Stubo and the last time the stub was used Clickingthe scenario name shows the stored matchers and responses for the scenario
Also available is a command file scratchpad for entering one or more commands For example one could enter
endsessionsession=first_sessiondeletestubsscenario=firstendsessionsession=second_sessiondeletestubsscenario=second
All the above commands would then be executed See the results on the tracker page
The manage page also shows stubs delay policies and any external modules loaded
13
Stub-O-Matic Documentation Release 0X beta
53 Analytics
The lsquoAnalyticsrsquo tab takes you to a dashboard showing statistics about the Stubo server The orange bar along the topallows you to select different Stubo servers and time periods The first of the 4 squares on the right turns on a legend
Charts
chart title descriptionstubssecondresponse rate
number of stubs requested per second
delay added user defined delays to stubs these can be specified to emulate response times of thesystem being stubbed
latency average time taken to return a stub excluding network and user defined delaysrequestresponse size size in kb of the requests to stubo and responses deliveredhost response latency where Stubo is deployed across multiple servers this chart compares the different servers
against each otherfailure rate percent of failed requests (HTTP response gt= 400)health checks Stubo has a regular health check ping to demonstrate that all systems are working
properlyrunning putstubcount
cumulative total of stubs loaded
running responsecount
cumulative total of stubs retrieved
Also note the Stubo API documentation for getstats
14 Chapter 5 GUI
CHAPTER 6
Integration with Test Tools
Stub-O-Matic works with all major software test execution tools Stubo of course does not run tests or evaluateresults it emulates back-end systems needed by tests
Data dependent software tests typically consist of 3 steps
1 Data setup
2 Test execution
3 Data teardown
Working with Stubo to provide back-end data is no different
Stuborsquos API can be used to setup or teardown data directly Stubo command files offer a shorthand for using the APIand are much more compact Any test tool that can make HTTP calls can use Stubo
61 Test steps example
1 Trigger an http call similar to this from the test tool of your choice
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_setupcommands
2 Run tests
3 Trigger a teardown http call similar to this from the test tool
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_teardowncommands
Setup would typically i) delete stubs ii) set any delays iii) submit a recording via a series of putstub calls and finallyiv) begin a session in playback
Teardown would typically i) end session ii) delete stubs
Setup and teardown can be performed with a mixture of command files (YAML or commands) and direct HTTP APIcalls to stubo
15
Stub-O-Matic Documentation Release 0X beta
16 Chapter 6 Integration with Test Tools
CHAPTER 7
Stubbing
71 Add Delay to a Response
To emulate response times of systems being stubbed one can set a delay on stubs Delays are set on the stub responsenot on the request to stubo This is because one request into the system being tested can result in several back-endstubocalls each of which might need different delay settings
Stubo provides reusable delay_policy objects which can be used across many stubs and across sessions The de-lay_policy can be altered on the fly during a loadperformance test if desired There are currently two delay_policytypes to choose from (1) fixed and (2)random with mean and standard deviation
A fixed delay policy can be created or modified with a commnand such as
putdelay_policyname=a_fixed_delay_policyampdelay_type=fixedampmilliseconds=200
Note that the value for the lsquonamersquo can be set to any string and is used again in the putstub A variable delay can be setup as follows
putdelay_policyname=a_normalvariate_delay_policyampdelay_type=normalvariateampmean=100ampstddev=50
Allowed lsquodelay typersquo values are lsquofixedrsquo and lsquonormalvariatersquo The next step is to use the delay policy when loadingstubs eg
putstubsession=abcampdelay_policy=a_fixed_delay_policy
bull Whenever the stub above is used the named delay policy will be applied
bull Delay policies may be altered on the fly during a test run if desired To change a delay on the fly simply repeatthe putdelay_policy command with a different value Typically this would be done directly by the test toolrather than from a command file
bull Delay policies can be used across multiple sessions This allows for easier alteration of policies where manysessions use the same policy It also means that you should name policies carefully and only alter or sharepolicies you are in control of
72 Matching
Stub-O-Matic currently supports various types of matchers
17
Stub-O-Matic Documentation Release 0X beta
73 Body contains matching
One or more matchers can be defined whereby all matchers must be contained in the request to return a specifiedresponse A typical difficulty with system requests is that they often contain superfluous information such as sessionIDs or time stamps that get in the way of matching a request to the desired response One way Stub-O-Matic can solvethis problem is by the use of lsquomatchersrsquo Matchers are parts of the request that matter to you only what is needed tofind the correct response file
For example a request may include
ltsessionidgt123456ltsessionidgtlttransactionidgt0987654321-9999lttransactionidgt
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Of these 4 lines only the last 2 matter the first 2 lines will differ from one request to the next and be set by the systemmaking the request This request then will be different each time it is made How can one easily match a request likethis to a canned response file without writing code for each requestresponse pair The solution used by Stub-O-Maticis to not store requestresponse file pairs but rather store matchers and the response In the case above the matcherswould be
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Stub-O-Matic contains code which will take a request as input search for the response with the best matchers Everymatcher must be contained in the request to be a match When a match is found the corresponding response is servedback
All matchers must be present in the request to make a match Stub-O-Matic will return the response from the firstmatch if there is more than one possible match Note that whitespace and carriage returns are ignored when matching
Alternatives to removing parts of the request you donrsquot want to match on are
1 using a matcher template to substitute ids or dates etc from the request into the matcher OR
2 using a user exit to run custom code A convenience class XMLManglerExit is provided for XML payloads toignore XMl elements andor attributes from the matching process
74 Templated Matcher
This is an example of a template matcher than extracts values from the request into the matcher The xmltree object isan lxml parsed root element passed into the template
ltrequestgtlttransidgtxmltreexpath(requesttransid)[0]textlttransidgtltdtgt2009-10-24T000000+0000ltdtgtltflightnumbergt0455ltflightnumbergtltrequestgt
75 Templated Responses
A hand crafted Stubo response may contain variables which are substituted in For example a response may contain
set flight_nbr=XX1234 The flight number is flight_nbr
18 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
The resulting response served wil be
The flight number is XX1234
Code may be used for example adding random elements to a response
import random set rnd_nbr = randomrandint(1200) Hello World 1000 + rnd_nbr
The resulting response served will be
Hello World - followed by a number between 1000 and 1200
To put todayrsquos date in a response include the following
today_strformat(dmy)
Stubo responses are run through a Tornado template and any logic or commands allowed in these templates can beused See httpwwwtornadoweborgenstabletemplatehtml for details
76 Request Data in Responses
761 Text Manipulation
It is possible to pull data from the stub request and put it in the response Say you have 200 stubs that are identicalapart from a userid They could be condensed into one stub which uses the userid from the request in the responserequest
ltuseridgtabc123ltuseridgt
matcher
ltuseridgt
response template (using Python)
Hello to request_text[814]
response
Hello to abc123
762 XML Manipulation
Sometimes it is easier to use XPATH to pull data from a request and place it in a response If you need the SessionIdfrom this request
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtJohn1234ltwbsSessionIdgt
Pull the SessionId from the request and use it within the response template
76 Request Data in Responses 19
Stub-O-Matic Documentation Release 0X beta
set namespaces = soapenv httpschemasxmlsoaporgsoapenvelopewbs httpxmlaaacomws200901WBS_Session-20xsd
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtxmltreexpath(soapenvHeaderwbsSessionwbsSessionId
namespaces=namespaces)[0]textltwbsSessionIdgt
Note the xmltree variable is the parsed xml request (as an lxml Root object) made available to the template if therequest is valid xml
Another example - no namespaces in the request Request
ltxml version=10 encoding=UTF-8gtltCompensateCustomersCheck APCWCustom=-1 LocalCurrencyCode=GBP NumberOfAdults=1gt
Response
set currency_code = xmltreexpath(CompensateCustomersCheck)[0]attrib[LocalCurrencyCode] ltresponse LocalCurrencyCode=currency_codegtpay meltresponsegt
77 Stateful Stubs
State is important in simulating back-end systems For example if one was to check-in passenger Bob on a particularflight the back-end would allow this request to succeed only one time and subsequent attempts to check-in the samepassenger on the same flight would be rejected Stubo can emulate this functionality or if desired for repeatable testsStubo can allow Bob to be checked in multiple times on the same flight
Stateful stubbing is enabled automatically If stubs are recorded or loaded in which the same request returns differentresponses Stubo will remember this and internally keep track of which response should come next
If your test is to check-in Bob multiple times simply do not load the stubs representing the check-in rejection andStubo will allow multiple check-ins
78 User Exits
Stub-O-Matic provides hooks into the runtime to execute custom user code Stubo can execute your custom code totransform stub matchers requests andor responses This is a powerful mechanism when your stubs contain dynamicdata or need to change over time This can be useful to perform lsquointelligent stubbingrsquo for example to roll dates Theuser exit API is contained in the stuboextuser_exit module
781 API
782 XML Auto Mangling
If your data is XML a convenience class XMLUserExit is provided to enable auto mangling
20 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
783 API
79 Hooks
710 Modules
Modules are added with the putmodule command and referenced via the ext_module url arg in the putstub call Forexample
putmodulename=staticcmdstestsextxsltmanglerpydeletestubsscenario=mangler_xsltbeginsessionscenario=mangler_xsltampsession=mangler_xslt1ampmode=recordputstubsession=mangler_xslt1ampext_module=mangler firstrequest firstresponseendsessionsession=mangler_xslt1
This example dynamically imports a test mangler from the Stubo server Typically these external python modules thathook into the Stubo runtime would be sourced from source control system using a URL
Note user exit processing is only enabled if the lsquoext_modulersquo variable is set to a pre-loaded module during the putstubcall Each Stub-O-Matic host URL has its own copy of any loaded modules Such modules are available to any testand as such should typically be loaded once per test suite
711 Splitting
Useful to removing random elements from matchers It can happen that the system under test will include seeminglyrandom elements in the request This will cause the next run of the test to not match This problem can be solved byprocessing matchers when loading - during a record A matcher can be split into two with the variable text left outFuture request must match both of the matchers to get the response An example is
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N11111234567806P1ltCommandgtltCryptic_GetScreen_Querygt
Which becomes two matchers
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N1111
and
06P1ltCommandgtltCryptic_GetScreen_Querygt
cutting out the random number lsquo12345678rsquo in the middle
This matcher splitting can be done manually or automatically with a user exit Seestaticcmdstestsextsplitters1firstcommands for and example It uses the rules module at staticcmdstestsextaaapy
Multiple matchers excluding random request elements can be created manually and loaded as such
putstubsession=splitter_record splitter_1textMatcher splitter_2textMatcher splitterresponse
712 Caching Values
If emulating back-end behaviour means writing some code for a particularly tricky behaviour Stubo exposes a simplekey-value cache API to matchers and responses
79 Hooks 21
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
CHAPTER 3
Stubo Command File
Command files are used to load stub files and add behaviour to them (state dates etc)
Stubs can also be loaded and tested using individual calls to the Stub-O-Matic server using the Stubo HTTP APICommand files are a shorthand for the API making it simpler to enable repeatable automated interactions with Stub-O-Matic by grouping a set of commands together required for a particular test
31 Command file (YAML)
An example
Stubo YAML Execution ordering is 1) commands 2) recording 3) playback
Commands go herecommands
-putmodule
varsname staticcmdstestsrestyamlnooppy
-putdelay_policy
varsname slowdelay_type fixedmilliseconds 1000
Describe your stubs hererecording
scenario restsession rest_recordingstubs-file stub1jsonvars
recorded_at as_date(2015-01-10)ext_module noop
-json
request method GETbodyPatterns [
7
Stub-O-Matic Documentation Release 0X beta
jsonpath [cmdx]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all ok 2headers Content-Type applicationjsonX-Custom-Header 1234
vars
foo bar
Provide your requests hereplayback
scenario restsession rest_playbackrequests-
file request1jsonvars
played_at as_date(2015-01-20)-
json method GETbody cmd x yheaders
Content-Type applicationjsonX-Custom-Header 1234
varsplayed_at as_date(2015-01-20)
Note the payloads can be defined either in place with the lsquojsonrsquo key or point to an external reference via the lsquofilersquo keyThe file can either reference a local statically served file as shown above or be a URI reference Typically it will be aURI reference to a location in a users source control system where their stubs are stored
The local file references in this example are shown below
stub1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
8 Chapter 3 Stubo Command File
Stub-O-Matic Documentation Release 0X beta
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
request1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
Note that these json payloads for the request and response are defined as strings Stubo also excepts the definition asdictionaries
32 Command Scripting
The YAML file is run through a template processor before being parsed and executed by Stubo Any variables definedsuch as lsquoplayed_atrsquo will evaluated and appropriate subsitutions made
A roll date example
(daterollyaml)
playbackrequests- file dateroll_1433930288_0requestresponse dateroll_1433930288_0stubo_responsevars
getresponse_arg this stub was played at 2015-06-10 095744839438
32 Command Scripting 9
Stub-O-Matic Documentation Release 0X beta
play_date 2014-09-12priority 1putstub_arg this stub was recorded at 2015-06-10 095744839387rec_date 2014-09-10tracking_level full
scenario daterollsession dateroll_1433930288
recordingscenario daterollsession dateroll_1433930288stubs- file dateroll_1433930288_0json
Referenced files
dateroll_1433930288_0json
priority 1args
priority 1rec_date 2014-09-10putstub_arg this stub was recorded at 2015-06-10 095744839387
request
bodyPatterns contains [
ltxml version=10 encoding=UTF-8 standalone=yes gtnltrollmegt n ltOriginDateTimegtroll_date(2014-09-10 as_date(rec_date) as_date(play_date))T000000ZltOriginDateTimegtnltrollmegt]
method POST
response
body ltresponsegtnltputstub_arggt raw putstub_arg ltputstub_arggtnltgetresponse_arggt getresponse_arg ltgetresponse_arggtnltresponsegtstatus 200
dateroll_1433930288_0request
body ltxml version=10 encoding=UTF-8 standalone=yesgtnltrollmegtn ltOriginDateTimegt2014-09-12T000000ZltOriginDateTimegtnltrollmegtheaders host nullpath nullquery uri nullmethod POST
10 Chapter 3 Stubo Command File
CHAPTER 4
Stubo Command File (text format)
The line orientated text version of the command file is an old format It does not support the matching required forREST For example you canrsquot match on the request headers or method It only supports one type of matcher whichis the lsquobody containsrsquo matcher If the text specified for the lsquobody containsrsquo matcher stub is found in the request amatch is found This is often however the only type of matcher required to successfully stub a system This format isstill supported for import via execcmds but will not be exported anymore If a text commands file is imported via alsquocommandsrsquo text file on export via getexport a lsquoyamlrsquo file will be exported
41 Text Command file
bull Lines beginning with a lsquorsquo are treated as comments
bull Commands follow the Stub-O-Matic REST API for example be-ginsessionscenario=abcampsession=12345ampmode=record
bull When the API requires something in the HTTP body input can be sourced by listing files after the command(separated by commas) Files are assumed to be in the same directory as the command file Alternatively a fullURI can be provided to source the input (matcher request or response)
bull Blank lines are ignored
bull An example command is putstubsession=12345rsquofeng_001textMatcher feng_002textMatcherfeng_001response A putstub command must have at least one matcher file and one response file Thisexample has two matchers
What goes in the command files
bull A textMatcher file could be some or all of the text from the request eg AB1234ltFlightInfoQuerygt
bull Multiple textMatchers files may be used in a stub All must evaluate to True against the request for a responseto be returned
bull request files contain the text of a request (eg text xml json) Note the using playback in commands is onlyused to test the stubs as these would be issued by the real system being tested in normal situations
bull response files contain the text of a response (eg text xml json)
bull Any naming convention can be used for matcher request and response files We have used suffixes oftextMatcher request amp response but you are free to use other extensions like matcherxml requestjson re-sponsexml etc
11
Stub-O-Matic Documentation Release 0X beta
42 Command Scripting
Stub-O-Matic supports the ability to load stubs from various and multiple sources This means people can organise allthe stubs that support a particular service or use case together and any test that needs those stubs could pull them infrom common stub libraries maximising stub reuse
set foreign_url = httpwwwgooglecouk set other_url = httpwwwsuncom deletestubsscenario=smartbeginsessionscenario=smartampsession=smart_1ampmode=recordputstubsession=smart_1url=other_urlurl=foreign_urlputstubsession=smart_1text=random_rubbletext=response_textendsessionsession=smart_1
Command files are also programmable with Python code snippets See the example below
set session_name = smart_1 deletestubsscenario=smartbeginsessionscenario=smartampsession=session_nameampmode=recordputstubsession=session_namefirsttextMatcherfirstresponseendsessionsession=session_namebeginsessionscenario=smartampsession=session_nameampmode=playback make 3 requests for i in range(03) getresponsesession=session_namefirstrequest end endsessionsession=session_name
43 Passing Arguments
Arguments can be passed into command files and used in them The following example allows the session and scenarioto be named externally to the command file
execcmdscmdFile=staticcmdstestsacceptsmart_argcommandsampscen=bobampsession=smart_1
The above call can be used as follows
deletestubsscenario=scen[0]beginsessionscenario=scen[0]ampsession= session[0] ampmode=recordputstubsession=session[0]firsttextMatcherfirstresponseendsessionsession=session[0]beginsessionscenario=scen[0]ampsession=session[0]ampmode=playback for i in range(03) getresponsesession=session[0]firstrequest end endsessionsession=session[0]
12 Chapter 4 Stubo Command File (text format)
CHAPTER 5
GUI
51 Tracker
The Stubo lsquoTrackerrsquo page shows details of all requests submitted to Stub-O-Matic and the received response Clickon the link in the function column to show a detailed view Debug level tracking can be added to a getresponse callif lsquotracking_level=fullrsquo is added to the call Matching errors will also trigger some extra debug data to be added to anormal track record
Stubo can not keep tracker data forever so it will eventually drop off the server as space is used up
The tracker page shows all Stubo actions performed on the server Filters are provided to filter on session datetimeresponse times and errors
To go into detail on a particular transaction click the link in the lsquoFunctionrsquo column or lsquoDetailsrsquo in the lsquoStubo Responsersquocolumn For a getresponse call the matchers and responses for the call can be found from the gtgtgt view scenario details(stub matchers and responses) link at the top of the details page
Filtering The tracker filter fields should be self explanatory By default the tracker page shows only data sent to theStubo URL shown at the top of the browser To see tracker items from other URLs on the same hardware tick thelsquoAll Hostsrsquo box
52 Manage
The manage page allows users to execute command files and view system state
The list of loaded stubs includes the date the stubs were loaded into Stubo and the last time the stub was used Clickingthe scenario name shows the stored matchers and responses for the scenario
Also available is a command file scratchpad for entering one or more commands For example one could enter
endsessionsession=first_sessiondeletestubsscenario=firstendsessionsession=second_sessiondeletestubsscenario=second
All the above commands would then be executed See the results on the tracker page
The manage page also shows stubs delay policies and any external modules loaded
13
Stub-O-Matic Documentation Release 0X beta
53 Analytics
The lsquoAnalyticsrsquo tab takes you to a dashboard showing statistics about the Stubo server The orange bar along the topallows you to select different Stubo servers and time periods The first of the 4 squares on the right turns on a legend
Charts
chart title descriptionstubssecondresponse rate
number of stubs requested per second
delay added user defined delays to stubs these can be specified to emulate response times of thesystem being stubbed
latency average time taken to return a stub excluding network and user defined delaysrequestresponse size size in kb of the requests to stubo and responses deliveredhost response latency where Stubo is deployed across multiple servers this chart compares the different servers
against each otherfailure rate percent of failed requests (HTTP response gt= 400)health checks Stubo has a regular health check ping to demonstrate that all systems are working
properlyrunning putstubcount
cumulative total of stubs loaded
running responsecount
cumulative total of stubs retrieved
Also note the Stubo API documentation for getstats
14 Chapter 5 GUI
CHAPTER 6
Integration with Test Tools
Stub-O-Matic works with all major software test execution tools Stubo of course does not run tests or evaluateresults it emulates back-end systems needed by tests
Data dependent software tests typically consist of 3 steps
1 Data setup
2 Test execution
3 Data teardown
Working with Stubo to provide back-end data is no different
Stuborsquos API can be used to setup or teardown data directly Stubo command files offer a shorthand for using the APIand are much more compact Any test tool that can make HTTP calls can use Stubo
61 Test steps example
1 Trigger an http call similar to this from the test tool of your choice
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_setupcommands
2 Run tests
3 Trigger a teardown http call similar to this from the test tool
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_teardowncommands
Setup would typically i) delete stubs ii) set any delays iii) submit a recording via a series of putstub calls and finallyiv) begin a session in playback
Teardown would typically i) end session ii) delete stubs
Setup and teardown can be performed with a mixture of command files (YAML or commands) and direct HTTP APIcalls to stubo
15
Stub-O-Matic Documentation Release 0X beta
16 Chapter 6 Integration with Test Tools
CHAPTER 7
Stubbing
71 Add Delay to a Response
To emulate response times of systems being stubbed one can set a delay on stubs Delays are set on the stub responsenot on the request to stubo This is because one request into the system being tested can result in several back-endstubocalls each of which might need different delay settings
Stubo provides reusable delay_policy objects which can be used across many stubs and across sessions The de-lay_policy can be altered on the fly during a loadperformance test if desired There are currently two delay_policytypes to choose from (1) fixed and (2)random with mean and standard deviation
A fixed delay policy can be created or modified with a commnand such as
putdelay_policyname=a_fixed_delay_policyampdelay_type=fixedampmilliseconds=200
Note that the value for the lsquonamersquo can be set to any string and is used again in the putstub A variable delay can be setup as follows
putdelay_policyname=a_normalvariate_delay_policyampdelay_type=normalvariateampmean=100ampstddev=50
Allowed lsquodelay typersquo values are lsquofixedrsquo and lsquonormalvariatersquo The next step is to use the delay policy when loadingstubs eg
putstubsession=abcampdelay_policy=a_fixed_delay_policy
bull Whenever the stub above is used the named delay policy will be applied
bull Delay policies may be altered on the fly during a test run if desired To change a delay on the fly simply repeatthe putdelay_policy command with a different value Typically this would be done directly by the test toolrather than from a command file
bull Delay policies can be used across multiple sessions This allows for easier alteration of policies where manysessions use the same policy It also means that you should name policies carefully and only alter or sharepolicies you are in control of
72 Matching
Stub-O-Matic currently supports various types of matchers
17
Stub-O-Matic Documentation Release 0X beta
73 Body contains matching
One or more matchers can be defined whereby all matchers must be contained in the request to return a specifiedresponse A typical difficulty with system requests is that they often contain superfluous information such as sessionIDs or time stamps that get in the way of matching a request to the desired response One way Stub-O-Matic can solvethis problem is by the use of lsquomatchersrsquo Matchers are parts of the request that matter to you only what is needed tofind the correct response file
For example a request may include
ltsessionidgt123456ltsessionidgtlttransactionidgt0987654321-9999lttransactionidgt
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Of these 4 lines only the last 2 matter the first 2 lines will differ from one request to the next and be set by the systemmaking the request This request then will be different each time it is made How can one easily match a request likethis to a canned response file without writing code for each requestresponse pair The solution used by Stub-O-Maticis to not store requestresponse file pairs but rather store matchers and the response In the case above the matcherswould be
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Stub-O-Matic contains code which will take a request as input search for the response with the best matchers Everymatcher must be contained in the request to be a match When a match is found the corresponding response is servedback
All matchers must be present in the request to make a match Stub-O-Matic will return the response from the firstmatch if there is more than one possible match Note that whitespace and carriage returns are ignored when matching
Alternatives to removing parts of the request you donrsquot want to match on are
1 using a matcher template to substitute ids or dates etc from the request into the matcher OR
2 using a user exit to run custom code A convenience class XMLManglerExit is provided for XML payloads toignore XMl elements andor attributes from the matching process
74 Templated Matcher
This is an example of a template matcher than extracts values from the request into the matcher The xmltree object isan lxml parsed root element passed into the template
ltrequestgtlttransidgtxmltreexpath(requesttransid)[0]textlttransidgtltdtgt2009-10-24T000000+0000ltdtgtltflightnumbergt0455ltflightnumbergtltrequestgt
75 Templated Responses
A hand crafted Stubo response may contain variables which are substituted in For example a response may contain
set flight_nbr=XX1234 The flight number is flight_nbr
18 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
The resulting response served wil be
The flight number is XX1234
Code may be used for example adding random elements to a response
import random set rnd_nbr = randomrandint(1200) Hello World 1000 + rnd_nbr
The resulting response served will be
Hello World - followed by a number between 1000 and 1200
To put todayrsquos date in a response include the following
today_strformat(dmy)
Stubo responses are run through a Tornado template and any logic or commands allowed in these templates can beused See httpwwwtornadoweborgenstabletemplatehtml for details
76 Request Data in Responses
761 Text Manipulation
It is possible to pull data from the stub request and put it in the response Say you have 200 stubs that are identicalapart from a userid They could be condensed into one stub which uses the userid from the request in the responserequest
ltuseridgtabc123ltuseridgt
matcher
ltuseridgt
response template (using Python)
Hello to request_text[814]
response
Hello to abc123
762 XML Manipulation
Sometimes it is easier to use XPATH to pull data from a request and place it in a response If you need the SessionIdfrom this request
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtJohn1234ltwbsSessionIdgt
Pull the SessionId from the request and use it within the response template
76 Request Data in Responses 19
Stub-O-Matic Documentation Release 0X beta
set namespaces = soapenv httpschemasxmlsoaporgsoapenvelopewbs httpxmlaaacomws200901WBS_Session-20xsd
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtxmltreexpath(soapenvHeaderwbsSessionwbsSessionId
namespaces=namespaces)[0]textltwbsSessionIdgt
Note the xmltree variable is the parsed xml request (as an lxml Root object) made available to the template if therequest is valid xml
Another example - no namespaces in the request Request
ltxml version=10 encoding=UTF-8gtltCompensateCustomersCheck APCWCustom=-1 LocalCurrencyCode=GBP NumberOfAdults=1gt
Response
set currency_code = xmltreexpath(CompensateCustomersCheck)[0]attrib[LocalCurrencyCode] ltresponse LocalCurrencyCode=currency_codegtpay meltresponsegt
77 Stateful Stubs
State is important in simulating back-end systems For example if one was to check-in passenger Bob on a particularflight the back-end would allow this request to succeed only one time and subsequent attempts to check-in the samepassenger on the same flight would be rejected Stubo can emulate this functionality or if desired for repeatable testsStubo can allow Bob to be checked in multiple times on the same flight
Stateful stubbing is enabled automatically If stubs are recorded or loaded in which the same request returns differentresponses Stubo will remember this and internally keep track of which response should come next
If your test is to check-in Bob multiple times simply do not load the stubs representing the check-in rejection andStubo will allow multiple check-ins
78 User Exits
Stub-O-Matic provides hooks into the runtime to execute custom user code Stubo can execute your custom code totransform stub matchers requests andor responses This is a powerful mechanism when your stubs contain dynamicdata or need to change over time This can be useful to perform lsquointelligent stubbingrsquo for example to roll dates Theuser exit API is contained in the stuboextuser_exit module
781 API
782 XML Auto Mangling
If your data is XML a convenience class XMLUserExit is provided to enable auto mangling
20 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
783 API
79 Hooks
710 Modules
Modules are added with the putmodule command and referenced via the ext_module url arg in the putstub call Forexample
putmodulename=staticcmdstestsextxsltmanglerpydeletestubsscenario=mangler_xsltbeginsessionscenario=mangler_xsltampsession=mangler_xslt1ampmode=recordputstubsession=mangler_xslt1ampext_module=mangler firstrequest firstresponseendsessionsession=mangler_xslt1
This example dynamically imports a test mangler from the Stubo server Typically these external python modules thathook into the Stubo runtime would be sourced from source control system using a URL
Note user exit processing is only enabled if the lsquoext_modulersquo variable is set to a pre-loaded module during the putstubcall Each Stub-O-Matic host URL has its own copy of any loaded modules Such modules are available to any testand as such should typically be loaded once per test suite
711 Splitting
Useful to removing random elements from matchers It can happen that the system under test will include seeminglyrandom elements in the request This will cause the next run of the test to not match This problem can be solved byprocessing matchers when loading - during a record A matcher can be split into two with the variable text left outFuture request must match both of the matchers to get the response An example is
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N11111234567806P1ltCommandgtltCryptic_GetScreen_Querygt
Which becomes two matchers
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N1111
and
06P1ltCommandgtltCryptic_GetScreen_Querygt
cutting out the random number lsquo12345678rsquo in the middle
This matcher splitting can be done manually or automatically with a user exit Seestaticcmdstestsextsplitters1firstcommands for and example It uses the rules module at staticcmdstestsextaaapy
Multiple matchers excluding random request elements can be created manually and loaded as such
putstubsession=splitter_record splitter_1textMatcher splitter_2textMatcher splitterresponse
712 Caching Values
If emulating back-end behaviour means writing some code for a particularly tricky behaviour Stubo exposes a simplekey-value cache API to matchers and responses
79 Hooks 21
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
Stub-O-Matic Documentation Release 0X beta
jsonpath [cmdx]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all ok 2headers Content-Type applicationjsonX-Custom-Header 1234
vars
foo bar
Provide your requests hereplayback
scenario restsession rest_playbackrequests-
file request1jsonvars
played_at as_date(2015-01-20)-
json method GETbody cmd x yheaders
Content-Type applicationjsonX-Custom-Header 1234
varsplayed_at as_date(2015-01-20)
Note the payloads can be defined either in place with the lsquojsonrsquo key or point to an external reference via the lsquofilersquo keyThe file can either reference a local statically served file as shown above or be a URI reference Typically it will be aURI reference to a location in a users source control system where their stubs are stored
The local file references in this example are shown below
stub1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
8 Chapter 3 Stubo Command File
Stub-O-Matic Documentation Release 0X beta
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
request1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
Note that these json payloads for the request and response are defined as strings Stubo also excepts the definition asdictionaries
32 Command Scripting
The YAML file is run through a template processor before being parsed and executed by Stubo Any variables definedsuch as lsquoplayed_atrsquo will evaluated and appropriate subsitutions made
A roll date example
(daterollyaml)
playbackrequests- file dateroll_1433930288_0requestresponse dateroll_1433930288_0stubo_responsevars
getresponse_arg this stub was played at 2015-06-10 095744839438
32 Command Scripting 9
Stub-O-Matic Documentation Release 0X beta
play_date 2014-09-12priority 1putstub_arg this stub was recorded at 2015-06-10 095744839387rec_date 2014-09-10tracking_level full
scenario daterollsession dateroll_1433930288
recordingscenario daterollsession dateroll_1433930288stubs- file dateroll_1433930288_0json
Referenced files
dateroll_1433930288_0json
priority 1args
priority 1rec_date 2014-09-10putstub_arg this stub was recorded at 2015-06-10 095744839387
request
bodyPatterns contains [
ltxml version=10 encoding=UTF-8 standalone=yes gtnltrollmegt n ltOriginDateTimegtroll_date(2014-09-10 as_date(rec_date) as_date(play_date))T000000ZltOriginDateTimegtnltrollmegt]
method POST
response
body ltresponsegtnltputstub_arggt raw putstub_arg ltputstub_arggtnltgetresponse_arggt getresponse_arg ltgetresponse_arggtnltresponsegtstatus 200
dateroll_1433930288_0request
body ltxml version=10 encoding=UTF-8 standalone=yesgtnltrollmegtn ltOriginDateTimegt2014-09-12T000000ZltOriginDateTimegtnltrollmegtheaders host nullpath nullquery uri nullmethod POST
10 Chapter 3 Stubo Command File
CHAPTER 4
Stubo Command File (text format)
The line orientated text version of the command file is an old format It does not support the matching required forREST For example you canrsquot match on the request headers or method It only supports one type of matcher whichis the lsquobody containsrsquo matcher If the text specified for the lsquobody containsrsquo matcher stub is found in the request amatch is found This is often however the only type of matcher required to successfully stub a system This format isstill supported for import via execcmds but will not be exported anymore If a text commands file is imported via alsquocommandsrsquo text file on export via getexport a lsquoyamlrsquo file will be exported
41 Text Command file
bull Lines beginning with a lsquorsquo are treated as comments
bull Commands follow the Stub-O-Matic REST API for example be-ginsessionscenario=abcampsession=12345ampmode=record
bull When the API requires something in the HTTP body input can be sourced by listing files after the command(separated by commas) Files are assumed to be in the same directory as the command file Alternatively a fullURI can be provided to source the input (matcher request or response)
bull Blank lines are ignored
bull An example command is putstubsession=12345rsquofeng_001textMatcher feng_002textMatcherfeng_001response A putstub command must have at least one matcher file and one response file Thisexample has two matchers
What goes in the command files
bull A textMatcher file could be some or all of the text from the request eg AB1234ltFlightInfoQuerygt
bull Multiple textMatchers files may be used in a stub All must evaluate to True against the request for a responseto be returned
bull request files contain the text of a request (eg text xml json) Note the using playback in commands is onlyused to test the stubs as these would be issued by the real system being tested in normal situations
bull response files contain the text of a response (eg text xml json)
bull Any naming convention can be used for matcher request and response files We have used suffixes oftextMatcher request amp response but you are free to use other extensions like matcherxml requestjson re-sponsexml etc
11
Stub-O-Matic Documentation Release 0X beta
42 Command Scripting
Stub-O-Matic supports the ability to load stubs from various and multiple sources This means people can organise allthe stubs that support a particular service or use case together and any test that needs those stubs could pull them infrom common stub libraries maximising stub reuse
set foreign_url = httpwwwgooglecouk set other_url = httpwwwsuncom deletestubsscenario=smartbeginsessionscenario=smartampsession=smart_1ampmode=recordputstubsession=smart_1url=other_urlurl=foreign_urlputstubsession=smart_1text=random_rubbletext=response_textendsessionsession=smart_1
Command files are also programmable with Python code snippets See the example below
set session_name = smart_1 deletestubsscenario=smartbeginsessionscenario=smartampsession=session_nameampmode=recordputstubsession=session_namefirsttextMatcherfirstresponseendsessionsession=session_namebeginsessionscenario=smartampsession=session_nameampmode=playback make 3 requests for i in range(03) getresponsesession=session_namefirstrequest end endsessionsession=session_name
43 Passing Arguments
Arguments can be passed into command files and used in them The following example allows the session and scenarioto be named externally to the command file
execcmdscmdFile=staticcmdstestsacceptsmart_argcommandsampscen=bobampsession=smart_1
The above call can be used as follows
deletestubsscenario=scen[0]beginsessionscenario=scen[0]ampsession= session[0] ampmode=recordputstubsession=session[0]firsttextMatcherfirstresponseendsessionsession=session[0]beginsessionscenario=scen[0]ampsession=session[0]ampmode=playback for i in range(03) getresponsesession=session[0]firstrequest end endsessionsession=session[0]
12 Chapter 4 Stubo Command File (text format)
CHAPTER 5
GUI
51 Tracker
The Stubo lsquoTrackerrsquo page shows details of all requests submitted to Stub-O-Matic and the received response Clickon the link in the function column to show a detailed view Debug level tracking can be added to a getresponse callif lsquotracking_level=fullrsquo is added to the call Matching errors will also trigger some extra debug data to be added to anormal track record
Stubo can not keep tracker data forever so it will eventually drop off the server as space is used up
The tracker page shows all Stubo actions performed on the server Filters are provided to filter on session datetimeresponse times and errors
To go into detail on a particular transaction click the link in the lsquoFunctionrsquo column or lsquoDetailsrsquo in the lsquoStubo Responsersquocolumn For a getresponse call the matchers and responses for the call can be found from the gtgtgt view scenario details(stub matchers and responses) link at the top of the details page
Filtering The tracker filter fields should be self explanatory By default the tracker page shows only data sent to theStubo URL shown at the top of the browser To see tracker items from other URLs on the same hardware tick thelsquoAll Hostsrsquo box
52 Manage
The manage page allows users to execute command files and view system state
The list of loaded stubs includes the date the stubs were loaded into Stubo and the last time the stub was used Clickingthe scenario name shows the stored matchers and responses for the scenario
Also available is a command file scratchpad for entering one or more commands For example one could enter
endsessionsession=first_sessiondeletestubsscenario=firstendsessionsession=second_sessiondeletestubsscenario=second
All the above commands would then be executed See the results on the tracker page
The manage page also shows stubs delay policies and any external modules loaded
13
Stub-O-Matic Documentation Release 0X beta
53 Analytics
The lsquoAnalyticsrsquo tab takes you to a dashboard showing statistics about the Stubo server The orange bar along the topallows you to select different Stubo servers and time periods The first of the 4 squares on the right turns on a legend
Charts
chart title descriptionstubssecondresponse rate
number of stubs requested per second
delay added user defined delays to stubs these can be specified to emulate response times of thesystem being stubbed
latency average time taken to return a stub excluding network and user defined delaysrequestresponse size size in kb of the requests to stubo and responses deliveredhost response latency where Stubo is deployed across multiple servers this chart compares the different servers
against each otherfailure rate percent of failed requests (HTTP response gt= 400)health checks Stubo has a regular health check ping to demonstrate that all systems are working
properlyrunning putstubcount
cumulative total of stubs loaded
running responsecount
cumulative total of stubs retrieved
Also note the Stubo API documentation for getstats
14 Chapter 5 GUI
CHAPTER 6
Integration with Test Tools
Stub-O-Matic works with all major software test execution tools Stubo of course does not run tests or evaluateresults it emulates back-end systems needed by tests
Data dependent software tests typically consist of 3 steps
1 Data setup
2 Test execution
3 Data teardown
Working with Stubo to provide back-end data is no different
Stuborsquos API can be used to setup or teardown data directly Stubo command files offer a shorthand for using the APIand are much more compact Any test tool that can make HTTP calls can use Stubo
61 Test steps example
1 Trigger an http call similar to this from the test tool of your choice
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_setupcommands
2 Run tests
3 Trigger a teardown http call similar to this from the test tool
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_teardowncommands
Setup would typically i) delete stubs ii) set any delays iii) submit a recording via a series of putstub calls and finallyiv) begin a session in playback
Teardown would typically i) end session ii) delete stubs
Setup and teardown can be performed with a mixture of command files (YAML or commands) and direct HTTP APIcalls to stubo
15
Stub-O-Matic Documentation Release 0X beta
16 Chapter 6 Integration with Test Tools
CHAPTER 7
Stubbing
71 Add Delay to a Response
To emulate response times of systems being stubbed one can set a delay on stubs Delays are set on the stub responsenot on the request to stubo This is because one request into the system being tested can result in several back-endstubocalls each of which might need different delay settings
Stubo provides reusable delay_policy objects which can be used across many stubs and across sessions The de-lay_policy can be altered on the fly during a loadperformance test if desired There are currently two delay_policytypes to choose from (1) fixed and (2)random with mean and standard deviation
A fixed delay policy can be created or modified with a commnand such as
putdelay_policyname=a_fixed_delay_policyampdelay_type=fixedampmilliseconds=200
Note that the value for the lsquonamersquo can be set to any string and is used again in the putstub A variable delay can be setup as follows
putdelay_policyname=a_normalvariate_delay_policyampdelay_type=normalvariateampmean=100ampstddev=50
Allowed lsquodelay typersquo values are lsquofixedrsquo and lsquonormalvariatersquo The next step is to use the delay policy when loadingstubs eg
putstubsession=abcampdelay_policy=a_fixed_delay_policy
bull Whenever the stub above is used the named delay policy will be applied
bull Delay policies may be altered on the fly during a test run if desired To change a delay on the fly simply repeatthe putdelay_policy command with a different value Typically this would be done directly by the test toolrather than from a command file
bull Delay policies can be used across multiple sessions This allows for easier alteration of policies where manysessions use the same policy It also means that you should name policies carefully and only alter or sharepolicies you are in control of
72 Matching
Stub-O-Matic currently supports various types of matchers
17
Stub-O-Matic Documentation Release 0X beta
73 Body contains matching
One or more matchers can be defined whereby all matchers must be contained in the request to return a specifiedresponse A typical difficulty with system requests is that they often contain superfluous information such as sessionIDs or time stamps that get in the way of matching a request to the desired response One way Stub-O-Matic can solvethis problem is by the use of lsquomatchersrsquo Matchers are parts of the request that matter to you only what is needed tofind the correct response file
For example a request may include
ltsessionidgt123456ltsessionidgtlttransactionidgt0987654321-9999lttransactionidgt
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Of these 4 lines only the last 2 matter the first 2 lines will differ from one request to the next and be set by the systemmaking the request This request then will be different each time it is made How can one easily match a request likethis to a canned response file without writing code for each requestresponse pair The solution used by Stub-O-Maticis to not store requestresponse file pairs but rather store matchers and the response In the case above the matcherswould be
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Stub-O-Matic contains code which will take a request as input search for the response with the best matchers Everymatcher must be contained in the request to be a match When a match is found the corresponding response is servedback
All matchers must be present in the request to make a match Stub-O-Matic will return the response from the firstmatch if there is more than one possible match Note that whitespace and carriage returns are ignored when matching
Alternatives to removing parts of the request you donrsquot want to match on are
1 using a matcher template to substitute ids or dates etc from the request into the matcher OR
2 using a user exit to run custom code A convenience class XMLManglerExit is provided for XML payloads toignore XMl elements andor attributes from the matching process
74 Templated Matcher
This is an example of a template matcher than extracts values from the request into the matcher The xmltree object isan lxml parsed root element passed into the template
ltrequestgtlttransidgtxmltreexpath(requesttransid)[0]textlttransidgtltdtgt2009-10-24T000000+0000ltdtgtltflightnumbergt0455ltflightnumbergtltrequestgt
75 Templated Responses
A hand crafted Stubo response may contain variables which are substituted in For example a response may contain
set flight_nbr=XX1234 The flight number is flight_nbr
18 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
The resulting response served wil be
The flight number is XX1234
Code may be used for example adding random elements to a response
import random set rnd_nbr = randomrandint(1200) Hello World 1000 + rnd_nbr
The resulting response served will be
Hello World - followed by a number between 1000 and 1200
To put todayrsquos date in a response include the following
today_strformat(dmy)
Stubo responses are run through a Tornado template and any logic or commands allowed in these templates can beused See httpwwwtornadoweborgenstabletemplatehtml for details
76 Request Data in Responses
761 Text Manipulation
It is possible to pull data from the stub request and put it in the response Say you have 200 stubs that are identicalapart from a userid They could be condensed into one stub which uses the userid from the request in the responserequest
ltuseridgtabc123ltuseridgt
matcher
ltuseridgt
response template (using Python)
Hello to request_text[814]
response
Hello to abc123
762 XML Manipulation
Sometimes it is easier to use XPATH to pull data from a request and place it in a response If you need the SessionIdfrom this request
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtJohn1234ltwbsSessionIdgt
Pull the SessionId from the request and use it within the response template
76 Request Data in Responses 19
Stub-O-Matic Documentation Release 0X beta
set namespaces = soapenv httpschemasxmlsoaporgsoapenvelopewbs httpxmlaaacomws200901WBS_Session-20xsd
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtxmltreexpath(soapenvHeaderwbsSessionwbsSessionId
namespaces=namespaces)[0]textltwbsSessionIdgt
Note the xmltree variable is the parsed xml request (as an lxml Root object) made available to the template if therequest is valid xml
Another example - no namespaces in the request Request
ltxml version=10 encoding=UTF-8gtltCompensateCustomersCheck APCWCustom=-1 LocalCurrencyCode=GBP NumberOfAdults=1gt
Response
set currency_code = xmltreexpath(CompensateCustomersCheck)[0]attrib[LocalCurrencyCode] ltresponse LocalCurrencyCode=currency_codegtpay meltresponsegt
77 Stateful Stubs
State is important in simulating back-end systems For example if one was to check-in passenger Bob on a particularflight the back-end would allow this request to succeed only one time and subsequent attempts to check-in the samepassenger on the same flight would be rejected Stubo can emulate this functionality or if desired for repeatable testsStubo can allow Bob to be checked in multiple times on the same flight
Stateful stubbing is enabled automatically If stubs are recorded or loaded in which the same request returns differentresponses Stubo will remember this and internally keep track of which response should come next
If your test is to check-in Bob multiple times simply do not load the stubs representing the check-in rejection andStubo will allow multiple check-ins
78 User Exits
Stub-O-Matic provides hooks into the runtime to execute custom user code Stubo can execute your custom code totransform stub matchers requests andor responses This is a powerful mechanism when your stubs contain dynamicdata or need to change over time This can be useful to perform lsquointelligent stubbingrsquo for example to roll dates Theuser exit API is contained in the stuboextuser_exit module
781 API
782 XML Auto Mangling
If your data is XML a convenience class XMLUserExit is provided to enable auto mangling
20 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
783 API
79 Hooks
710 Modules
Modules are added with the putmodule command and referenced via the ext_module url arg in the putstub call Forexample
putmodulename=staticcmdstestsextxsltmanglerpydeletestubsscenario=mangler_xsltbeginsessionscenario=mangler_xsltampsession=mangler_xslt1ampmode=recordputstubsession=mangler_xslt1ampext_module=mangler firstrequest firstresponseendsessionsession=mangler_xslt1
This example dynamically imports a test mangler from the Stubo server Typically these external python modules thathook into the Stubo runtime would be sourced from source control system using a URL
Note user exit processing is only enabled if the lsquoext_modulersquo variable is set to a pre-loaded module during the putstubcall Each Stub-O-Matic host URL has its own copy of any loaded modules Such modules are available to any testand as such should typically be loaded once per test suite
711 Splitting
Useful to removing random elements from matchers It can happen that the system under test will include seeminglyrandom elements in the request This will cause the next run of the test to not match This problem can be solved byprocessing matchers when loading - during a record A matcher can be split into two with the variable text left outFuture request must match both of the matchers to get the response An example is
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N11111234567806P1ltCommandgtltCryptic_GetScreen_Querygt
Which becomes two matchers
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N1111
and
06P1ltCommandgtltCryptic_GetScreen_Querygt
cutting out the random number lsquo12345678rsquo in the middle
This matcher splitting can be done manually or automatically with a user exit Seestaticcmdstestsextsplitters1firstcommands for and example It uses the rules module at staticcmdstestsextaaapy
Multiple matchers excluding random request elements can be created manually and loaded as such
putstubsession=splitter_record splitter_1textMatcher splitter_2textMatcher splitterresponse
712 Caching Values
If emulating back-end behaviour means writing some code for a particularly tricky behaviour Stubo exposes a simplekey-value cache API to matchers and responses
79 Hooks 21
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
Stub-O-Matic Documentation Release 0X beta
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
request1json
request
method GETbodyPatterns [
jsonpath [cmda]
]headers
Content-Type applicationjsonX-Custom-Header 1234
response
status 200body version 123 data status all okheaders Content-Type applicationjsonX-Custom-Header 1234
Note that these json payloads for the request and response are defined as strings Stubo also excepts the definition asdictionaries
32 Command Scripting
The YAML file is run through a template processor before being parsed and executed by Stubo Any variables definedsuch as lsquoplayed_atrsquo will evaluated and appropriate subsitutions made
A roll date example
(daterollyaml)
playbackrequests- file dateroll_1433930288_0requestresponse dateroll_1433930288_0stubo_responsevars
getresponse_arg this stub was played at 2015-06-10 095744839438
32 Command Scripting 9
Stub-O-Matic Documentation Release 0X beta
play_date 2014-09-12priority 1putstub_arg this stub was recorded at 2015-06-10 095744839387rec_date 2014-09-10tracking_level full
scenario daterollsession dateroll_1433930288
recordingscenario daterollsession dateroll_1433930288stubs- file dateroll_1433930288_0json
Referenced files
dateroll_1433930288_0json
priority 1args
priority 1rec_date 2014-09-10putstub_arg this stub was recorded at 2015-06-10 095744839387
request
bodyPatterns contains [
ltxml version=10 encoding=UTF-8 standalone=yes gtnltrollmegt n ltOriginDateTimegtroll_date(2014-09-10 as_date(rec_date) as_date(play_date))T000000ZltOriginDateTimegtnltrollmegt]
method POST
response
body ltresponsegtnltputstub_arggt raw putstub_arg ltputstub_arggtnltgetresponse_arggt getresponse_arg ltgetresponse_arggtnltresponsegtstatus 200
dateroll_1433930288_0request
body ltxml version=10 encoding=UTF-8 standalone=yesgtnltrollmegtn ltOriginDateTimegt2014-09-12T000000ZltOriginDateTimegtnltrollmegtheaders host nullpath nullquery uri nullmethod POST
10 Chapter 3 Stubo Command File
CHAPTER 4
Stubo Command File (text format)
The line orientated text version of the command file is an old format It does not support the matching required forREST For example you canrsquot match on the request headers or method It only supports one type of matcher whichis the lsquobody containsrsquo matcher If the text specified for the lsquobody containsrsquo matcher stub is found in the request amatch is found This is often however the only type of matcher required to successfully stub a system This format isstill supported for import via execcmds but will not be exported anymore If a text commands file is imported via alsquocommandsrsquo text file on export via getexport a lsquoyamlrsquo file will be exported
41 Text Command file
bull Lines beginning with a lsquorsquo are treated as comments
bull Commands follow the Stub-O-Matic REST API for example be-ginsessionscenario=abcampsession=12345ampmode=record
bull When the API requires something in the HTTP body input can be sourced by listing files after the command(separated by commas) Files are assumed to be in the same directory as the command file Alternatively a fullURI can be provided to source the input (matcher request or response)
bull Blank lines are ignored
bull An example command is putstubsession=12345rsquofeng_001textMatcher feng_002textMatcherfeng_001response A putstub command must have at least one matcher file and one response file Thisexample has two matchers
What goes in the command files
bull A textMatcher file could be some or all of the text from the request eg AB1234ltFlightInfoQuerygt
bull Multiple textMatchers files may be used in a stub All must evaluate to True against the request for a responseto be returned
bull request files contain the text of a request (eg text xml json) Note the using playback in commands is onlyused to test the stubs as these would be issued by the real system being tested in normal situations
bull response files contain the text of a response (eg text xml json)
bull Any naming convention can be used for matcher request and response files We have used suffixes oftextMatcher request amp response but you are free to use other extensions like matcherxml requestjson re-sponsexml etc
11
Stub-O-Matic Documentation Release 0X beta
42 Command Scripting
Stub-O-Matic supports the ability to load stubs from various and multiple sources This means people can organise allthe stubs that support a particular service or use case together and any test that needs those stubs could pull them infrom common stub libraries maximising stub reuse
set foreign_url = httpwwwgooglecouk set other_url = httpwwwsuncom deletestubsscenario=smartbeginsessionscenario=smartampsession=smart_1ampmode=recordputstubsession=smart_1url=other_urlurl=foreign_urlputstubsession=smart_1text=random_rubbletext=response_textendsessionsession=smart_1
Command files are also programmable with Python code snippets See the example below
set session_name = smart_1 deletestubsscenario=smartbeginsessionscenario=smartampsession=session_nameampmode=recordputstubsession=session_namefirsttextMatcherfirstresponseendsessionsession=session_namebeginsessionscenario=smartampsession=session_nameampmode=playback make 3 requests for i in range(03) getresponsesession=session_namefirstrequest end endsessionsession=session_name
43 Passing Arguments
Arguments can be passed into command files and used in them The following example allows the session and scenarioto be named externally to the command file
execcmdscmdFile=staticcmdstestsacceptsmart_argcommandsampscen=bobampsession=smart_1
The above call can be used as follows
deletestubsscenario=scen[0]beginsessionscenario=scen[0]ampsession= session[0] ampmode=recordputstubsession=session[0]firsttextMatcherfirstresponseendsessionsession=session[0]beginsessionscenario=scen[0]ampsession=session[0]ampmode=playback for i in range(03) getresponsesession=session[0]firstrequest end endsessionsession=session[0]
12 Chapter 4 Stubo Command File (text format)
CHAPTER 5
GUI
51 Tracker
The Stubo lsquoTrackerrsquo page shows details of all requests submitted to Stub-O-Matic and the received response Clickon the link in the function column to show a detailed view Debug level tracking can be added to a getresponse callif lsquotracking_level=fullrsquo is added to the call Matching errors will also trigger some extra debug data to be added to anormal track record
Stubo can not keep tracker data forever so it will eventually drop off the server as space is used up
The tracker page shows all Stubo actions performed on the server Filters are provided to filter on session datetimeresponse times and errors
To go into detail on a particular transaction click the link in the lsquoFunctionrsquo column or lsquoDetailsrsquo in the lsquoStubo Responsersquocolumn For a getresponse call the matchers and responses for the call can be found from the gtgtgt view scenario details(stub matchers and responses) link at the top of the details page
Filtering The tracker filter fields should be self explanatory By default the tracker page shows only data sent to theStubo URL shown at the top of the browser To see tracker items from other URLs on the same hardware tick thelsquoAll Hostsrsquo box
52 Manage
The manage page allows users to execute command files and view system state
The list of loaded stubs includes the date the stubs were loaded into Stubo and the last time the stub was used Clickingthe scenario name shows the stored matchers and responses for the scenario
Also available is a command file scratchpad for entering one or more commands For example one could enter
endsessionsession=first_sessiondeletestubsscenario=firstendsessionsession=second_sessiondeletestubsscenario=second
All the above commands would then be executed See the results on the tracker page
The manage page also shows stubs delay policies and any external modules loaded
13
Stub-O-Matic Documentation Release 0X beta
53 Analytics
The lsquoAnalyticsrsquo tab takes you to a dashboard showing statistics about the Stubo server The orange bar along the topallows you to select different Stubo servers and time periods The first of the 4 squares on the right turns on a legend
Charts
chart title descriptionstubssecondresponse rate
number of stubs requested per second
delay added user defined delays to stubs these can be specified to emulate response times of thesystem being stubbed
latency average time taken to return a stub excluding network and user defined delaysrequestresponse size size in kb of the requests to stubo and responses deliveredhost response latency where Stubo is deployed across multiple servers this chart compares the different servers
against each otherfailure rate percent of failed requests (HTTP response gt= 400)health checks Stubo has a regular health check ping to demonstrate that all systems are working
properlyrunning putstubcount
cumulative total of stubs loaded
running responsecount
cumulative total of stubs retrieved
Also note the Stubo API documentation for getstats
14 Chapter 5 GUI
CHAPTER 6
Integration with Test Tools
Stub-O-Matic works with all major software test execution tools Stubo of course does not run tests or evaluateresults it emulates back-end systems needed by tests
Data dependent software tests typically consist of 3 steps
1 Data setup
2 Test execution
3 Data teardown
Working with Stubo to provide back-end data is no different
Stuborsquos API can be used to setup or teardown data directly Stubo command files offer a shorthand for using the APIand are much more compact Any test tool that can make HTTP calls can use Stubo
61 Test steps example
1 Trigger an http call similar to this from the test tool of your choice
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_setupcommands
2 Run tests
3 Trigger a teardown http call similar to this from the test tool
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_teardowncommands
Setup would typically i) delete stubs ii) set any delays iii) submit a recording via a series of putstub calls and finallyiv) begin a session in playback
Teardown would typically i) end session ii) delete stubs
Setup and teardown can be performed with a mixture of command files (YAML or commands) and direct HTTP APIcalls to stubo
15
Stub-O-Matic Documentation Release 0X beta
16 Chapter 6 Integration with Test Tools
CHAPTER 7
Stubbing
71 Add Delay to a Response
To emulate response times of systems being stubbed one can set a delay on stubs Delays are set on the stub responsenot on the request to stubo This is because one request into the system being tested can result in several back-endstubocalls each of which might need different delay settings
Stubo provides reusable delay_policy objects which can be used across many stubs and across sessions The de-lay_policy can be altered on the fly during a loadperformance test if desired There are currently two delay_policytypes to choose from (1) fixed and (2)random with mean and standard deviation
A fixed delay policy can be created or modified with a commnand such as
putdelay_policyname=a_fixed_delay_policyampdelay_type=fixedampmilliseconds=200
Note that the value for the lsquonamersquo can be set to any string and is used again in the putstub A variable delay can be setup as follows
putdelay_policyname=a_normalvariate_delay_policyampdelay_type=normalvariateampmean=100ampstddev=50
Allowed lsquodelay typersquo values are lsquofixedrsquo and lsquonormalvariatersquo The next step is to use the delay policy when loadingstubs eg
putstubsession=abcampdelay_policy=a_fixed_delay_policy
bull Whenever the stub above is used the named delay policy will be applied
bull Delay policies may be altered on the fly during a test run if desired To change a delay on the fly simply repeatthe putdelay_policy command with a different value Typically this would be done directly by the test toolrather than from a command file
bull Delay policies can be used across multiple sessions This allows for easier alteration of policies where manysessions use the same policy It also means that you should name policies carefully and only alter or sharepolicies you are in control of
72 Matching
Stub-O-Matic currently supports various types of matchers
17
Stub-O-Matic Documentation Release 0X beta
73 Body contains matching
One or more matchers can be defined whereby all matchers must be contained in the request to return a specifiedresponse A typical difficulty with system requests is that they often contain superfluous information such as sessionIDs or time stamps that get in the way of matching a request to the desired response One way Stub-O-Matic can solvethis problem is by the use of lsquomatchersrsquo Matchers are parts of the request that matter to you only what is needed tofind the correct response file
For example a request may include
ltsessionidgt123456ltsessionidgtlttransactionidgt0987654321-9999lttransactionidgt
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Of these 4 lines only the last 2 matter the first 2 lines will differ from one request to the next and be set by the systemmaking the request This request then will be different each time it is made How can one easily match a request likethis to a canned response file without writing code for each requestresponse pair The solution used by Stub-O-Maticis to not store requestresponse file pairs but rather store matchers and the response In the case above the matcherswould be
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Stub-O-Matic contains code which will take a request as input search for the response with the best matchers Everymatcher must be contained in the request to be a match When a match is found the corresponding response is servedback
All matchers must be present in the request to make a match Stub-O-Matic will return the response from the firstmatch if there is more than one possible match Note that whitespace and carriage returns are ignored when matching
Alternatives to removing parts of the request you donrsquot want to match on are
1 using a matcher template to substitute ids or dates etc from the request into the matcher OR
2 using a user exit to run custom code A convenience class XMLManglerExit is provided for XML payloads toignore XMl elements andor attributes from the matching process
74 Templated Matcher
This is an example of a template matcher than extracts values from the request into the matcher The xmltree object isan lxml parsed root element passed into the template
ltrequestgtlttransidgtxmltreexpath(requesttransid)[0]textlttransidgtltdtgt2009-10-24T000000+0000ltdtgtltflightnumbergt0455ltflightnumbergtltrequestgt
75 Templated Responses
A hand crafted Stubo response may contain variables which are substituted in For example a response may contain
set flight_nbr=XX1234 The flight number is flight_nbr
18 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
The resulting response served wil be
The flight number is XX1234
Code may be used for example adding random elements to a response
import random set rnd_nbr = randomrandint(1200) Hello World 1000 + rnd_nbr
The resulting response served will be
Hello World - followed by a number between 1000 and 1200
To put todayrsquos date in a response include the following
today_strformat(dmy)
Stubo responses are run through a Tornado template and any logic or commands allowed in these templates can beused See httpwwwtornadoweborgenstabletemplatehtml for details
76 Request Data in Responses
761 Text Manipulation
It is possible to pull data from the stub request and put it in the response Say you have 200 stubs that are identicalapart from a userid They could be condensed into one stub which uses the userid from the request in the responserequest
ltuseridgtabc123ltuseridgt
matcher
ltuseridgt
response template (using Python)
Hello to request_text[814]
response
Hello to abc123
762 XML Manipulation
Sometimes it is easier to use XPATH to pull data from a request and place it in a response If you need the SessionIdfrom this request
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtJohn1234ltwbsSessionIdgt
Pull the SessionId from the request and use it within the response template
76 Request Data in Responses 19
Stub-O-Matic Documentation Release 0X beta
set namespaces = soapenv httpschemasxmlsoaporgsoapenvelopewbs httpxmlaaacomws200901WBS_Session-20xsd
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtxmltreexpath(soapenvHeaderwbsSessionwbsSessionId
namespaces=namespaces)[0]textltwbsSessionIdgt
Note the xmltree variable is the parsed xml request (as an lxml Root object) made available to the template if therequest is valid xml
Another example - no namespaces in the request Request
ltxml version=10 encoding=UTF-8gtltCompensateCustomersCheck APCWCustom=-1 LocalCurrencyCode=GBP NumberOfAdults=1gt
Response
set currency_code = xmltreexpath(CompensateCustomersCheck)[0]attrib[LocalCurrencyCode] ltresponse LocalCurrencyCode=currency_codegtpay meltresponsegt
77 Stateful Stubs
State is important in simulating back-end systems For example if one was to check-in passenger Bob on a particularflight the back-end would allow this request to succeed only one time and subsequent attempts to check-in the samepassenger on the same flight would be rejected Stubo can emulate this functionality or if desired for repeatable testsStubo can allow Bob to be checked in multiple times on the same flight
Stateful stubbing is enabled automatically If stubs are recorded or loaded in which the same request returns differentresponses Stubo will remember this and internally keep track of which response should come next
If your test is to check-in Bob multiple times simply do not load the stubs representing the check-in rejection andStubo will allow multiple check-ins
78 User Exits
Stub-O-Matic provides hooks into the runtime to execute custom user code Stubo can execute your custom code totransform stub matchers requests andor responses This is a powerful mechanism when your stubs contain dynamicdata or need to change over time This can be useful to perform lsquointelligent stubbingrsquo for example to roll dates Theuser exit API is contained in the stuboextuser_exit module
781 API
782 XML Auto Mangling
If your data is XML a convenience class XMLUserExit is provided to enable auto mangling
20 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
783 API
79 Hooks
710 Modules
Modules are added with the putmodule command and referenced via the ext_module url arg in the putstub call Forexample
putmodulename=staticcmdstestsextxsltmanglerpydeletestubsscenario=mangler_xsltbeginsessionscenario=mangler_xsltampsession=mangler_xslt1ampmode=recordputstubsession=mangler_xslt1ampext_module=mangler firstrequest firstresponseendsessionsession=mangler_xslt1
This example dynamically imports a test mangler from the Stubo server Typically these external python modules thathook into the Stubo runtime would be sourced from source control system using a URL
Note user exit processing is only enabled if the lsquoext_modulersquo variable is set to a pre-loaded module during the putstubcall Each Stub-O-Matic host URL has its own copy of any loaded modules Such modules are available to any testand as such should typically be loaded once per test suite
711 Splitting
Useful to removing random elements from matchers It can happen that the system under test will include seeminglyrandom elements in the request This will cause the next run of the test to not match This problem can be solved byprocessing matchers when loading - during a record A matcher can be split into two with the variable text left outFuture request must match both of the matchers to get the response An example is
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N11111234567806P1ltCommandgtltCryptic_GetScreen_Querygt
Which becomes two matchers
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N1111
and
06P1ltCommandgtltCryptic_GetScreen_Querygt
cutting out the random number lsquo12345678rsquo in the middle
This matcher splitting can be done manually or automatically with a user exit Seestaticcmdstestsextsplitters1firstcommands for and example It uses the rules module at staticcmdstestsextaaapy
Multiple matchers excluding random request elements can be created manually and loaded as such
putstubsession=splitter_record splitter_1textMatcher splitter_2textMatcher splitterresponse
712 Caching Values
If emulating back-end behaviour means writing some code for a particularly tricky behaviour Stubo exposes a simplekey-value cache API to matchers and responses
79 Hooks 21
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
Stub-O-Matic Documentation Release 0X beta
play_date 2014-09-12priority 1putstub_arg this stub was recorded at 2015-06-10 095744839387rec_date 2014-09-10tracking_level full
scenario daterollsession dateroll_1433930288
recordingscenario daterollsession dateroll_1433930288stubs- file dateroll_1433930288_0json
Referenced files
dateroll_1433930288_0json
priority 1args
priority 1rec_date 2014-09-10putstub_arg this stub was recorded at 2015-06-10 095744839387
request
bodyPatterns contains [
ltxml version=10 encoding=UTF-8 standalone=yes gtnltrollmegt n ltOriginDateTimegtroll_date(2014-09-10 as_date(rec_date) as_date(play_date))T000000ZltOriginDateTimegtnltrollmegt]
method POST
response
body ltresponsegtnltputstub_arggt raw putstub_arg ltputstub_arggtnltgetresponse_arggt getresponse_arg ltgetresponse_arggtnltresponsegtstatus 200
dateroll_1433930288_0request
body ltxml version=10 encoding=UTF-8 standalone=yesgtnltrollmegtn ltOriginDateTimegt2014-09-12T000000ZltOriginDateTimegtnltrollmegtheaders host nullpath nullquery uri nullmethod POST
10 Chapter 3 Stubo Command File
CHAPTER 4
Stubo Command File (text format)
The line orientated text version of the command file is an old format It does not support the matching required forREST For example you canrsquot match on the request headers or method It only supports one type of matcher whichis the lsquobody containsrsquo matcher If the text specified for the lsquobody containsrsquo matcher stub is found in the request amatch is found This is often however the only type of matcher required to successfully stub a system This format isstill supported for import via execcmds but will not be exported anymore If a text commands file is imported via alsquocommandsrsquo text file on export via getexport a lsquoyamlrsquo file will be exported
41 Text Command file
bull Lines beginning with a lsquorsquo are treated as comments
bull Commands follow the Stub-O-Matic REST API for example be-ginsessionscenario=abcampsession=12345ampmode=record
bull When the API requires something in the HTTP body input can be sourced by listing files after the command(separated by commas) Files are assumed to be in the same directory as the command file Alternatively a fullURI can be provided to source the input (matcher request or response)
bull Blank lines are ignored
bull An example command is putstubsession=12345rsquofeng_001textMatcher feng_002textMatcherfeng_001response A putstub command must have at least one matcher file and one response file Thisexample has two matchers
What goes in the command files
bull A textMatcher file could be some or all of the text from the request eg AB1234ltFlightInfoQuerygt
bull Multiple textMatchers files may be used in a stub All must evaluate to True against the request for a responseto be returned
bull request files contain the text of a request (eg text xml json) Note the using playback in commands is onlyused to test the stubs as these would be issued by the real system being tested in normal situations
bull response files contain the text of a response (eg text xml json)
bull Any naming convention can be used for matcher request and response files We have used suffixes oftextMatcher request amp response but you are free to use other extensions like matcherxml requestjson re-sponsexml etc
11
Stub-O-Matic Documentation Release 0X beta
42 Command Scripting
Stub-O-Matic supports the ability to load stubs from various and multiple sources This means people can organise allthe stubs that support a particular service or use case together and any test that needs those stubs could pull them infrom common stub libraries maximising stub reuse
set foreign_url = httpwwwgooglecouk set other_url = httpwwwsuncom deletestubsscenario=smartbeginsessionscenario=smartampsession=smart_1ampmode=recordputstubsession=smart_1url=other_urlurl=foreign_urlputstubsession=smart_1text=random_rubbletext=response_textendsessionsession=smart_1
Command files are also programmable with Python code snippets See the example below
set session_name = smart_1 deletestubsscenario=smartbeginsessionscenario=smartampsession=session_nameampmode=recordputstubsession=session_namefirsttextMatcherfirstresponseendsessionsession=session_namebeginsessionscenario=smartampsession=session_nameampmode=playback make 3 requests for i in range(03) getresponsesession=session_namefirstrequest end endsessionsession=session_name
43 Passing Arguments
Arguments can be passed into command files and used in them The following example allows the session and scenarioto be named externally to the command file
execcmdscmdFile=staticcmdstestsacceptsmart_argcommandsampscen=bobampsession=smart_1
The above call can be used as follows
deletestubsscenario=scen[0]beginsessionscenario=scen[0]ampsession= session[0] ampmode=recordputstubsession=session[0]firsttextMatcherfirstresponseendsessionsession=session[0]beginsessionscenario=scen[0]ampsession=session[0]ampmode=playback for i in range(03) getresponsesession=session[0]firstrequest end endsessionsession=session[0]
12 Chapter 4 Stubo Command File (text format)
CHAPTER 5
GUI
51 Tracker
The Stubo lsquoTrackerrsquo page shows details of all requests submitted to Stub-O-Matic and the received response Clickon the link in the function column to show a detailed view Debug level tracking can be added to a getresponse callif lsquotracking_level=fullrsquo is added to the call Matching errors will also trigger some extra debug data to be added to anormal track record
Stubo can not keep tracker data forever so it will eventually drop off the server as space is used up
The tracker page shows all Stubo actions performed on the server Filters are provided to filter on session datetimeresponse times and errors
To go into detail on a particular transaction click the link in the lsquoFunctionrsquo column or lsquoDetailsrsquo in the lsquoStubo Responsersquocolumn For a getresponse call the matchers and responses for the call can be found from the gtgtgt view scenario details(stub matchers and responses) link at the top of the details page
Filtering The tracker filter fields should be self explanatory By default the tracker page shows only data sent to theStubo URL shown at the top of the browser To see tracker items from other URLs on the same hardware tick thelsquoAll Hostsrsquo box
52 Manage
The manage page allows users to execute command files and view system state
The list of loaded stubs includes the date the stubs were loaded into Stubo and the last time the stub was used Clickingthe scenario name shows the stored matchers and responses for the scenario
Also available is a command file scratchpad for entering one or more commands For example one could enter
endsessionsession=first_sessiondeletestubsscenario=firstendsessionsession=second_sessiondeletestubsscenario=second
All the above commands would then be executed See the results on the tracker page
The manage page also shows stubs delay policies and any external modules loaded
13
Stub-O-Matic Documentation Release 0X beta
53 Analytics
The lsquoAnalyticsrsquo tab takes you to a dashboard showing statistics about the Stubo server The orange bar along the topallows you to select different Stubo servers and time periods The first of the 4 squares on the right turns on a legend
Charts
chart title descriptionstubssecondresponse rate
number of stubs requested per second
delay added user defined delays to stubs these can be specified to emulate response times of thesystem being stubbed
latency average time taken to return a stub excluding network and user defined delaysrequestresponse size size in kb of the requests to stubo and responses deliveredhost response latency where Stubo is deployed across multiple servers this chart compares the different servers
against each otherfailure rate percent of failed requests (HTTP response gt= 400)health checks Stubo has a regular health check ping to demonstrate that all systems are working
properlyrunning putstubcount
cumulative total of stubs loaded
running responsecount
cumulative total of stubs retrieved
Also note the Stubo API documentation for getstats
14 Chapter 5 GUI
CHAPTER 6
Integration with Test Tools
Stub-O-Matic works with all major software test execution tools Stubo of course does not run tests or evaluateresults it emulates back-end systems needed by tests
Data dependent software tests typically consist of 3 steps
1 Data setup
2 Test execution
3 Data teardown
Working with Stubo to provide back-end data is no different
Stuborsquos API can be used to setup or teardown data directly Stubo command files offer a shorthand for using the APIand are much more compact Any test tool that can make HTTP calls can use Stubo
61 Test steps example
1 Trigger an http call similar to this from the test tool of your choice
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_setupcommands
2 Run tests
3 Trigger a teardown http call similar to this from the test tool
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_teardowncommands
Setup would typically i) delete stubs ii) set any delays iii) submit a recording via a series of putstub calls and finallyiv) begin a session in playback
Teardown would typically i) end session ii) delete stubs
Setup and teardown can be performed with a mixture of command files (YAML or commands) and direct HTTP APIcalls to stubo
15
Stub-O-Matic Documentation Release 0X beta
16 Chapter 6 Integration with Test Tools
CHAPTER 7
Stubbing
71 Add Delay to a Response
To emulate response times of systems being stubbed one can set a delay on stubs Delays are set on the stub responsenot on the request to stubo This is because one request into the system being tested can result in several back-endstubocalls each of which might need different delay settings
Stubo provides reusable delay_policy objects which can be used across many stubs and across sessions The de-lay_policy can be altered on the fly during a loadperformance test if desired There are currently two delay_policytypes to choose from (1) fixed and (2)random with mean and standard deviation
A fixed delay policy can be created or modified with a commnand such as
putdelay_policyname=a_fixed_delay_policyampdelay_type=fixedampmilliseconds=200
Note that the value for the lsquonamersquo can be set to any string and is used again in the putstub A variable delay can be setup as follows
putdelay_policyname=a_normalvariate_delay_policyampdelay_type=normalvariateampmean=100ampstddev=50
Allowed lsquodelay typersquo values are lsquofixedrsquo and lsquonormalvariatersquo The next step is to use the delay policy when loadingstubs eg
putstubsession=abcampdelay_policy=a_fixed_delay_policy
bull Whenever the stub above is used the named delay policy will be applied
bull Delay policies may be altered on the fly during a test run if desired To change a delay on the fly simply repeatthe putdelay_policy command with a different value Typically this would be done directly by the test toolrather than from a command file
bull Delay policies can be used across multiple sessions This allows for easier alteration of policies where manysessions use the same policy It also means that you should name policies carefully and only alter or sharepolicies you are in control of
72 Matching
Stub-O-Matic currently supports various types of matchers
17
Stub-O-Matic Documentation Release 0X beta
73 Body contains matching
One or more matchers can be defined whereby all matchers must be contained in the request to return a specifiedresponse A typical difficulty with system requests is that they often contain superfluous information such as sessionIDs or time stamps that get in the way of matching a request to the desired response One way Stub-O-Matic can solvethis problem is by the use of lsquomatchersrsquo Matchers are parts of the request that matter to you only what is needed tofind the correct response file
For example a request may include
ltsessionidgt123456ltsessionidgtlttransactionidgt0987654321-9999lttransactionidgt
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Of these 4 lines only the last 2 matter the first 2 lines will differ from one request to the next and be set by the systemmaking the request This request then will be different each time it is made How can one easily match a request likethis to a canned response file without writing code for each requestresponse pair The solution used by Stub-O-Maticis to not store requestresponse file pairs but rather store matchers and the response In the case above the matcherswould be
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Stub-O-Matic contains code which will take a request as input search for the response with the best matchers Everymatcher must be contained in the request to be a match When a match is found the corresponding response is servedback
All matchers must be present in the request to make a match Stub-O-Matic will return the response from the firstmatch if there is more than one possible match Note that whitespace and carriage returns are ignored when matching
Alternatives to removing parts of the request you donrsquot want to match on are
1 using a matcher template to substitute ids or dates etc from the request into the matcher OR
2 using a user exit to run custom code A convenience class XMLManglerExit is provided for XML payloads toignore XMl elements andor attributes from the matching process
74 Templated Matcher
This is an example of a template matcher than extracts values from the request into the matcher The xmltree object isan lxml parsed root element passed into the template
ltrequestgtlttransidgtxmltreexpath(requesttransid)[0]textlttransidgtltdtgt2009-10-24T000000+0000ltdtgtltflightnumbergt0455ltflightnumbergtltrequestgt
75 Templated Responses
A hand crafted Stubo response may contain variables which are substituted in For example a response may contain
set flight_nbr=XX1234 The flight number is flight_nbr
18 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
The resulting response served wil be
The flight number is XX1234
Code may be used for example adding random elements to a response
import random set rnd_nbr = randomrandint(1200) Hello World 1000 + rnd_nbr
The resulting response served will be
Hello World - followed by a number between 1000 and 1200
To put todayrsquos date in a response include the following
today_strformat(dmy)
Stubo responses are run through a Tornado template and any logic or commands allowed in these templates can beused See httpwwwtornadoweborgenstabletemplatehtml for details
76 Request Data in Responses
761 Text Manipulation
It is possible to pull data from the stub request and put it in the response Say you have 200 stubs that are identicalapart from a userid They could be condensed into one stub which uses the userid from the request in the responserequest
ltuseridgtabc123ltuseridgt
matcher
ltuseridgt
response template (using Python)
Hello to request_text[814]
response
Hello to abc123
762 XML Manipulation
Sometimes it is easier to use XPATH to pull data from a request and place it in a response If you need the SessionIdfrom this request
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtJohn1234ltwbsSessionIdgt
Pull the SessionId from the request and use it within the response template
76 Request Data in Responses 19
Stub-O-Matic Documentation Release 0X beta
set namespaces = soapenv httpschemasxmlsoaporgsoapenvelopewbs httpxmlaaacomws200901WBS_Session-20xsd
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtxmltreexpath(soapenvHeaderwbsSessionwbsSessionId
namespaces=namespaces)[0]textltwbsSessionIdgt
Note the xmltree variable is the parsed xml request (as an lxml Root object) made available to the template if therequest is valid xml
Another example - no namespaces in the request Request
ltxml version=10 encoding=UTF-8gtltCompensateCustomersCheck APCWCustom=-1 LocalCurrencyCode=GBP NumberOfAdults=1gt
Response
set currency_code = xmltreexpath(CompensateCustomersCheck)[0]attrib[LocalCurrencyCode] ltresponse LocalCurrencyCode=currency_codegtpay meltresponsegt
77 Stateful Stubs
State is important in simulating back-end systems For example if one was to check-in passenger Bob on a particularflight the back-end would allow this request to succeed only one time and subsequent attempts to check-in the samepassenger on the same flight would be rejected Stubo can emulate this functionality or if desired for repeatable testsStubo can allow Bob to be checked in multiple times on the same flight
Stateful stubbing is enabled automatically If stubs are recorded or loaded in which the same request returns differentresponses Stubo will remember this and internally keep track of which response should come next
If your test is to check-in Bob multiple times simply do not load the stubs representing the check-in rejection andStubo will allow multiple check-ins
78 User Exits
Stub-O-Matic provides hooks into the runtime to execute custom user code Stubo can execute your custom code totransform stub matchers requests andor responses This is a powerful mechanism when your stubs contain dynamicdata or need to change over time This can be useful to perform lsquointelligent stubbingrsquo for example to roll dates Theuser exit API is contained in the stuboextuser_exit module
781 API
782 XML Auto Mangling
If your data is XML a convenience class XMLUserExit is provided to enable auto mangling
20 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
783 API
79 Hooks
710 Modules
Modules are added with the putmodule command and referenced via the ext_module url arg in the putstub call Forexample
putmodulename=staticcmdstestsextxsltmanglerpydeletestubsscenario=mangler_xsltbeginsessionscenario=mangler_xsltampsession=mangler_xslt1ampmode=recordputstubsession=mangler_xslt1ampext_module=mangler firstrequest firstresponseendsessionsession=mangler_xslt1
This example dynamically imports a test mangler from the Stubo server Typically these external python modules thathook into the Stubo runtime would be sourced from source control system using a URL
Note user exit processing is only enabled if the lsquoext_modulersquo variable is set to a pre-loaded module during the putstubcall Each Stub-O-Matic host URL has its own copy of any loaded modules Such modules are available to any testand as such should typically be loaded once per test suite
711 Splitting
Useful to removing random elements from matchers It can happen that the system under test will include seeminglyrandom elements in the request This will cause the next run of the test to not match This problem can be solved byprocessing matchers when loading - during a record A matcher can be split into two with the variable text left outFuture request must match both of the matchers to get the response An example is
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N11111234567806P1ltCommandgtltCryptic_GetScreen_Querygt
Which becomes two matchers
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N1111
and
06P1ltCommandgtltCryptic_GetScreen_Querygt
cutting out the random number lsquo12345678rsquo in the middle
This matcher splitting can be done manually or automatically with a user exit Seestaticcmdstestsextsplitters1firstcommands for and example It uses the rules module at staticcmdstestsextaaapy
Multiple matchers excluding random request elements can be created manually and loaded as such
putstubsession=splitter_record splitter_1textMatcher splitter_2textMatcher splitterresponse
712 Caching Values
If emulating back-end behaviour means writing some code for a particularly tricky behaviour Stubo exposes a simplekey-value cache API to matchers and responses
79 Hooks 21
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
CHAPTER 4
Stubo Command File (text format)
The line orientated text version of the command file is an old format It does not support the matching required forREST For example you canrsquot match on the request headers or method It only supports one type of matcher whichis the lsquobody containsrsquo matcher If the text specified for the lsquobody containsrsquo matcher stub is found in the request amatch is found This is often however the only type of matcher required to successfully stub a system This format isstill supported for import via execcmds but will not be exported anymore If a text commands file is imported via alsquocommandsrsquo text file on export via getexport a lsquoyamlrsquo file will be exported
41 Text Command file
bull Lines beginning with a lsquorsquo are treated as comments
bull Commands follow the Stub-O-Matic REST API for example be-ginsessionscenario=abcampsession=12345ampmode=record
bull When the API requires something in the HTTP body input can be sourced by listing files after the command(separated by commas) Files are assumed to be in the same directory as the command file Alternatively a fullURI can be provided to source the input (matcher request or response)
bull Blank lines are ignored
bull An example command is putstubsession=12345rsquofeng_001textMatcher feng_002textMatcherfeng_001response A putstub command must have at least one matcher file and one response file Thisexample has two matchers
What goes in the command files
bull A textMatcher file could be some or all of the text from the request eg AB1234ltFlightInfoQuerygt
bull Multiple textMatchers files may be used in a stub All must evaluate to True against the request for a responseto be returned
bull request files contain the text of a request (eg text xml json) Note the using playback in commands is onlyused to test the stubs as these would be issued by the real system being tested in normal situations
bull response files contain the text of a response (eg text xml json)
bull Any naming convention can be used for matcher request and response files We have used suffixes oftextMatcher request amp response but you are free to use other extensions like matcherxml requestjson re-sponsexml etc
11
Stub-O-Matic Documentation Release 0X beta
42 Command Scripting
Stub-O-Matic supports the ability to load stubs from various and multiple sources This means people can organise allthe stubs that support a particular service or use case together and any test that needs those stubs could pull them infrom common stub libraries maximising stub reuse
set foreign_url = httpwwwgooglecouk set other_url = httpwwwsuncom deletestubsscenario=smartbeginsessionscenario=smartampsession=smart_1ampmode=recordputstubsession=smart_1url=other_urlurl=foreign_urlputstubsession=smart_1text=random_rubbletext=response_textendsessionsession=smart_1
Command files are also programmable with Python code snippets See the example below
set session_name = smart_1 deletestubsscenario=smartbeginsessionscenario=smartampsession=session_nameampmode=recordputstubsession=session_namefirsttextMatcherfirstresponseendsessionsession=session_namebeginsessionscenario=smartampsession=session_nameampmode=playback make 3 requests for i in range(03) getresponsesession=session_namefirstrequest end endsessionsession=session_name
43 Passing Arguments
Arguments can be passed into command files and used in them The following example allows the session and scenarioto be named externally to the command file
execcmdscmdFile=staticcmdstestsacceptsmart_argcommandsampscen=bobampsession=smart_1
The above call can be used as follows
deletestubsscenario=scen[0]beginsessionscenario=scen[0]ampsession= session[0] ampmode=recordputstubsession=session[0]firsttextMatcherfirstresponseendsessionsession=session[0]beginsessionscenario=scen[0]ampsession=session[0]ampmode=playback for i in range(03) getresponsesession=session[0]firstrequest end endsessionsession=session[0]
12 Chapter 4 Stubo Command File (text format)
CHAPTER 5
GUI
51 Tracker
The Stubo lsquoTrackerrsquo page shows details of all requests submitted to Stub-O-Matic and the received response Clickon the link in the function column to show a detailed view Debug level tracking can be added to a getresponse callif lsquotracking_level=fullrsquo is added to the call Matching errors will also trigger some extra debug data to be added to anormal track record
Stubo can not keep tracker data forever so it will eventually drop off the server as space is used up
The tracker page shows all Stubo actions performed on the server Filters are provided to filter on session datetimeresponse times and errors
To go into detail on a particular transaction click the link in the lsquoFunctionrsquo column or lsquoDetailsrsquo in the lsquoStubo Responsersquocolumn For a getresponse call the matchers and responses for the call can be found from the gtgtgt view scenario details(stub matchers and responses) link at the top of the details page
Filtering The tracker filter fields should be self explanatory By default the tracker page shows only data sent to theStubo URL shown at the top of the browser To see tracker items from other URLs on the same hardware tick thelsquoAll Hostsrsquo box
52 Manage
The manage page allows users to execute command files and view system state
The list of loaded stubs includes the date the stubs were loaded into Stubo and the last time the stub was used Clickingthe scenario name shows the stored matchers and responses for the scenario
Also available is a command file scratchpad for entering one or more commands For example one could enter
endsessionsession=first_sessiondeletestubsscenario=firstendsessionsession=second_sessiondeletestubsscenario=second
All the above commands would then be executed See the results on the tracker page
The manage page also shows stubs delay policies and any external modules loaded
13
Stub-O-Matic Documentation Release 0X beta
53 Analytics
The lsquoAnalyticsrsquo tab takes you to a dashboard showing statistics about the Stubo server The orange bar along the topallows you to select different Stubo servers and time periods The first of the 4 squares on the right turns on a legend
Charts
chart title descriptionstubssecondresponse rate
number of stubs requested per second
delay added user defined delays to stubs these can be specified to emulate response times of thesystem being stubbed
latency average time taken to return a stub excluding network and user defined delaysrequestresponse size size in kb of the requests to stubo and responses deliveredhost response latency where Stubo is deployed across multiple servers this chart compares the different servers
against each otherfailure rate percent of failed requests (HTTP response gt= 400)health checks Stubo has a regular health check ping to demonstrate that all systems are working
properlyrunning putstubcount
cumulative total of stubs loaded
running responsecount
cumulative total of stubs retrieved
Also note the Stubo API documentation for getstats
14 Chapter 5 GUI
CHAPTER 6
Integration with Test Tools
Stub-O-Matic works with all major software test execution tools Stubo of course does not run tests or evaluateresults it emulates back-end systems needed by tests
Data dependent software tests typically consist of 3 steps
1 Data setup
2 Test execution
3 Data teardown
Working with Stubo to provide back-end data is no different
Stuborsquos API can be used to setup or teardown data directly Stubo command files offer a shorthand for using the APIand are much more compact Any test tool that can make HTTP calls can use Stubo
61 Test steps example
1 Trigger an http call similar to this from the test tool of your choice
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_setupcommands
2 Run tests
3 Trigger a teardown http call similar to this from the test tool
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_teardowncommands
Setup would typically i) delete stubs ii) set any delays iii) submit a recording via a series of putstub calls and finallyiv) begin a session in playback
Teardown would typically i) end session ii) delete stubs
Setup and teardown can be performed with a mixture of command files (YAML or commands) and direct HTTP APIcalls to stubo
15
Stub-O-Matic Documentation Release 0X beta
16 Chapter 6 Integration with Test Tools
CHAPTER 7
Stubbing
71 Add Delay to a Response
To emulate response times of systems being stubbed one can set a delay on stubs Delays are set on the stub responsenot on the request to stubo This is because one request into the system being tested can result in several back-endstubocalls each of which might need different delay settings
Stubo provides reusable delay_policy objects which can be used across many stubs and across sessions The de-lay_policy can be altered on the fly during a loadperformance test if desired There are currently two delay_policytypes to choose from (1) fixed and (2)random with mean and standard deviation
A fixed delay policy can be created or modified with a commnand such as
putdelay_policyname=a_fixed_delay_policyampdelay_type=fixedampmilliseconds=200
Note that the value for the lsquonamersquo can be set to any string and is used again in the putstub A variable delay can be setup as follows
putdelay_policyname=a_normalvariate_delay_policyampdelay_type=normalvariateampmean=100ampstddev=50
Allowed lsquodelay typersquo values are lsquofixedrsquo and lsquonormalvariatersquo The next step is to use the delay policy when loadingstubs eg
putstubsession=abcampdelay_policy=a_fixed_delay_policy
bull Whenever the stub above is used the named delay policy will be applied
bull Delay policies may be altered on the fly during a test run if desired To change a delay on the fly simply repeatthe putdelay_policy command with a different value Typically this would be done directly by the test toolrather than from a command file
bull Delay policies can be used across multiple sessions This allows for easier alteration of policies where manysessions use the same policy It also means that you should name policies carefully and only alter or sharepolicies you are in control of
72 Matching
Stub-O-Matic currently supports various types of matchers
17
Stub-O-Matic Documentation Release 0X beta
73 Body contains matching
One or more matchers can be defined whereby all matchers must be contained in the request to return a specifiedresponse A typical difficulty with system requests is that they often contain superfluous information such as sessionIDs or time stamps that get in the way of matching a request to the desired response One way Stub-O-Matic can solvethis problem is by the use of lsquomatchersrsquo Matchers are parts of the request that matter to you only what is needed tofind the correct response file
For example a request may include
ltsessionidgt123456ltsessionidgtlttransactionidgt0987654321-9999lttransactionidgt
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Of these 4 lines only the last 2 matter the first 2 lines will differ from one request to the next and be set by the systemmaking the request This request then will be different each time it is made How can one easily match a request likethis to a canned response file without writing code for each requestresponse pair The solution used by Stub-O-Maticis to not store requestresponse file pairs but rather store matchers and the response In the case above the matcherswould be
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Stub-O-Matic contains code which will take a request as input search for the response with the best matchers Everymatcher must be contained in the request to be a match When a match is found the corresponding response is servedback
All matchers must be present in the request to make a match Stub-O-Matic will return the response from the firstmatch if there is more than one possible match Note that whitespace and carriage returns are ignored when matching
Alternatives to removing parts of the request you donrsquot want to match on are
1 using a matcher template to substitute ids or dates etc from the request into the matcher OR
2 using a user exit to run custom code A convenience class XMLManglerExit is provided for XML payloads toignore XMl elements andor attributes from the matching process
74 Templated Matcher
This is an example of a template matcher than extracts values from the request into the matcher The xmltree object isan lxml parsed root element passed into the template
ltrequestgtlttransidgtxmltreexpath(requesttransid)[0]textlttransidgtltdtgt2009-10-24T000000+0000ltdtgtltflightnumbergt0455ltflightnumbergtltrequestgt
75 Templated Responses
A hand crafted Stubo response may contain variables which are substituted in For example a response may contain
set flight_nbr=XX1234 The flight number is flight_nbr
18 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
The resulting response served wil be
The flight number is XX1234
Code may be used for example adding random elements to a response
import random set rnd_nbr = randomrandint(1200) Hello World 1000 + rnd_nbr
The resulting response served will be
Hello World - followed by a number between 1000 and 1200
To put todayrsquos date in a response include the following
today_strformat(dmy)
Stubo responses are run through a Tornado template and any logic or commands allowed in these templates can beused See httpwwwtornadoweborgenstabletemplatehtml for details
76 Request Data in Responses
761 Text Manipulation
It is possible to pull data from the stub request and put it in the response Say you have 200 stubs that are identicalapart from a userid They could be condensed into one stub which uses the userid from the request in the responserequest
ltuseridgtabc123ltuseridgt
matcher
ltuseridgt
response template (using Python)
Hello to request_text[814]
response
Hello to abc123
762 XML Manipulation
Sometimes it is easier to use XPATH to pull data from a request and place it in a response If you need the SessionIdfrom this request
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtJohn1234ltwbsSessionIdgt
Pull the SessionId from the request and use it within the response template
76 Request Data in Responses 19
Stub-O-Matic Documentation Release 0X beta
set namespaces = soapenv httpschemasxmlsoaporgsoapenvelopewbs httpxmlaaacomws200901WBS_Session-20xsd
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtxmltreexpath(soapenvHeaderwbsSessionwbsSessionId
namespaces=namespaces)[0]textltwbsSessionIdgt
Note the xmltree variable is the parsed xml request (as an lxml Root object) made available to the template if therequest is valid xml
Another example - no namespaces in the request Request
ltxml version=10 encoding=UTF-8gtltCompensateCustomersCheck APCWCustom=-1 LocalCurrencyCode=GBP NumberOfAdults=1gt
Response
set currency_code = xmltreexpath(CompensateCustomersCheck)[0]attrib[LocalCurrencyCode] ltresponse LocalCurrencyCode=currency_codegtpay meltresponsegt
77 Stateful Stubs
State is important in simulating back-end systems For example if one was to check-in passenger Bob on a particularflight the back-end would allow this request to succeed only one time and subsequent attempts to check-in the samepassenger on the same flight would be rejected Stubo can emulate this functionality or if desired for repeatable testsStubo can allow Bob to be checked in multiple times on the same flight
Stateful stubbing is enabled automatically If stubs are recorded or loaded in which the same request returns differentresponses Stubo will remember this and internally keep track of which response should come next
If your test is to check-in Bob multiple times simply do not load the stubs representing the check-in rejection andStubo will allow multiple check-ins
78 User Exits
Stub-O-Matic provides hooks into the runtime to execute custom user code Stubo can execute your custom code totransform stub matchers requests andor responses This is a powerful mechanism when your stubs contain dynamicdata or need to change over time This can be useful to perform lsquointelligent stubbingrsquo for example to roll dates Theuser exit API is contained in the stuboextuser_exit module
781 API
782 XML Auto Mangling
If your data is XML a convenience class XMLUserExit is provided to enable auto mangling
20 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
783 API
79 Hooks
710 Modules
Modules are added with the putmodule command and referenced via the ext_module url arg in the putstub call Forexample
putmodulename=staticcmdstestsextxsltmanglerpydeletestubsscenario=mangler_xsltbeginsessionscenario=mangler_xsltampsession=mangler_xslt1ampmode=recordputstubsession=mangler_xslt1ampext_module=mangler firstrequest firstresponseendsessionsession=mangler_xslt1
This example dynamically imports a test mangler from the Stubo server Typically these external python modules thathook into the Stubo runtime would be sourced from source control system using a URL
Note user exit processing is only enabled if the lsquoext_modulersquo variable is set to a pre-loaded module during the putstubcall Each Stub-O-Matic host URL has its own copy of any loaded modules Such modules are available to any testand as such should typically be loaded once per test suite
711 Splitting
Useful to removing random elements from matchers It can happen that the system under test will include seeminglyrandom elements in the request This will cause the next run of the test to not match This problem can be solved byprocessing matchers when loading - during a record A matcher can be split into two with the variable text left outFuture request must match both of the matchers to get the response An example is
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N11111234567806P1ltCommandgtltCryptic_GetScreen_Querygt
Which becomes two matchers
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N1111
and
06P1ltCommandgtltCryptic_GetScreen_Querygt
cutting out the random number lsquo12345678rsquo in the middle
This matcher splitting can be done manually or automatically with a user exit Seestaticcmdstestsextsplitters1firstcommands for and example It uses the rules module at staticcmdstestsextaaapy
Multiple matchers excluding random request elements can be created manually and loaded as such
putstubsession=splitter_record splitter_1textMatcher splitter_2textMatcher splitterresponse
712 Caching Values
If emulating back-end behaviour means writing some code for a particularly tricky behaviour Stubo exposes a simplekey-value cache API to matchers and responses
79 Hooks 21
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
Stub-O-Matic Documentation Release 0X beta
42 Command Scripting
Stub-O-Matic supports the ability to load stubs from various and multiple sources This means people can organise allthe stubs that support a particular service or use case together and any test that needs those stubs could pull them infrom common stub libraries maximising stub reuse
set foreign_url = httpwwwgooglecouk set other_url = httpwwwsuncom deletestubsscenario=smartbeginsessionscenario=smartampsession=smart_1ampmode=recordputstubsession=smart_1url=other_urlurl=foreign_urlputstubsession=smart_1text=random_rubbletext=response_textendsessionsession=smart_1
Command files are also programmable with Python code snippets See the example below
set session_name = smart_1 deletestubsscenario=smartbeginsessionscenario=smartampsession=session_nameampmode=recordputstubsession=session_namefirsttextMatcherfirstresponseendsessionsession=session_namebeginsessionscenario=smartampsession=session_nameampmode=playback make 3 requests for i in range(03) getresponsesession=session_namefirstrequest end endsessionsession=session_name
43 Passing Arguments
Arguments can be passed into command files and used in them The following example allows the session and scenarioto be named externally to the command file
execcmdscmdFile=staticcmdstestsacceptsmart_argcommandsampscen=bobampsession=smart_1
The above call can be used as follows
deletestubsscenario=scen[0]beginsessionscenario=scen[0]ampsession= session[0] ampmode=recordputstubsession=session[0]firsttextMatcherfirstresponseendsessionsession=session[0]beginsessionscenario=scen[0]ampsession=session[0]ampmode=playback for i in range(03) getresponsesession=session[0]firstrequest end endsessionsession=session[0]
12 Chapter 4 Stubo Command File (text format)
CHAPTER 5
GUI
51 Tracker
The Stubo lsquoTrackerrsquo page shows details of all requests submitted to Stub-O-Matic and the received response Clickon the link in the function column to show a detailed view Debug level tracking can be added to a getresponse callif lsquotracking_level=fullrsquo is added to the call Matching errors will also trigger some extra debug data to be added to anormal track record
Stubo can not keep tracker data forever so it will eventually drop off the server as space is used up
The tracker page shows all Stubo actions performed on the server Filters are provided to filter on session datetimeresponse times and errors
To go into detail on a particular transaction click the link in the lsquoFunctionrsquo column or lsquoDetailsrsquo in the lsquoStubo Responsersquocolumn For a getresponse call the matchers and responses for the call can be found from the gtgtgt view scenario details(stub matchers and responses) link at the top of the details page
Filtering The tracker filter fields should be self explanatory By default the tracker page shows only data sent to theStubo URL shown at the top of the browser To see tracker items from other URLs on the same hardware tick thelsquoAll Hostsrsquo box
52 Manage
The manage page allows users to execute command files and view system state
The list of loaded stubs includes the date the stubs were loaded into Stubo and the last time the stub was used Clickingthe scenario name shows the stored matchers and responses for the scenario
Also available is a command file scratchpad for entering one or more commands For example one could enter
endsessionsession=first_sessiondeletestubsscenario=firstendsessionsession=second_sessiondeletestubsscenario=second
All the above commands would then be executed See the results on the tracker page
The manage page also shows stubs delay policies and any external modules loaded
13
Stub-O-Matic Documentation Release 0X beta
53 Analytics
The lsquoAnalyticsrsquo tab takes you to a dashboard showing statistics about the Stubo server The orange bar along the topallows you to select different Stubo servers and time periods The first of the 4 squares on the right turns on a legend
Charts
chart title descriptionstubssecondresponse rate
number of stubs requested per second
delay added user defined delays to stubs these can be specified to emulate response times of thesystem being stubbed
latency average time taken to return a stub excluding network and user defined delaysrequestresponse size size in kb of the requests to stubo and responses deliveredhost response latency where Stubo is deployed across multiple servers this chart compares the different servers
against each otherfailure rate percent of failed requests (HTTP response gt= 400)health checks Stubo has a regular health check ping to demonstrate that all systems are working
properlyrunning putstubcount
cumulative total of stubs loaded
running responsecount
cumulative total of stubs retrieved
Also note the Stubo API documentation for getstats
14 Chapter 5 GUI
CHAPTER 6
Integration with Test Tools
Stub-O-Matic works with all major software test execution tools Stubo of course does not run tests or evaluateresults it emulates back-end systems needed by tests
Data dependent software tests typically consist of 3 steps
1 Data setup
2 Test execution
3 Data teardown
Working with Stubo to provide back-end data is no different
Stuborsquos API can be used to setup or teardown data directly Stubo command files offer a shorthand for using the APIand are much more compact Any test tool that can make HTTP calls can use Stubo
61 Test steps example
1 Trigger an http call similar to this from the test tool of your choice
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_setupcommands
2 Run tests
3 Trigger a teardown http call similar to this from the test tool
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_teardowncommands
Setup would typically i) delete stubs ii) set any delays iii) submit a recording via a series of putstub calls and finallyiv) begin a session in playback
Teardown would typically i) end session ii) delete stubs
Setup and teardown can be performed with a mixture of command files (YAML or commands) and direct HTTP APIcalls to stubo
15
Stub-O-Matic Documentation Release 0X beta
16 Chapter 6 Integration with Test Tools
CHAPTER 7
Stubbing
71 Add Delay to a Response
To emulate response times of systems being stubbed one can set a delay on stubs Delays are set on the stub responsenot on the request to stubo This is because one request into the system being tested can result in several back-endstubocalls each of which might need different delay settings
Stubo provides reusable delay_policy objects which can be used across many stubs and across sessions The de-lay_policy can be altered on the fly during a loadperformance test if desired There are currently two delay_policytypes to choose from (1) fixed and (2)random with mean and standard deviation
A fixed delay policy can be created or modified with a commnand such as
putdelay_policyname=a_fixed_delay_policyampdelay_type=fixedampmilliseconds=200
Note that the value for the lsquonamersquo can be set to any string and is used again in the putstub A variable delay can be setup as follows
putdelay_policyname=a_normalvariate_delay_policyampdelay_type=normalvariateampmean=100ampstddev=50
Allowed lsquodelay typersquo values are lsquofixedrsquo and lsquonormalvariatersquo The next step is to use the delay policy when loadingstubs eg
putstubsession=abcampdelay_policy=a_fixed_delay_policy
bull Whenever the stub above is used the named delay policy will be applied
bull Delay policies may be altered on the fly during a test run if desired To change a delay on the fly simply repeatthe putdelay_policy command with a different value Typically this would be done directly by the test toolrather than from a command file
bull Delay policies can be used across multiple sessions This allows for easier alteration of policies where manysessions use the same policy It also means that you should name policies carefully and only alter or sharepolicies you are in control of
72 Matching
Stub-O-Matic currently supports various types of matchers
17
Stub-O-Matic Documentation Release 0X beta
73 Body contains matching
One or more matchers can be defined whereby all matchers must be contained in the request to return a specifiedresponse A typical difficulty with system requests is that they often contain superfluous information such as sessionIDs or time stamps that get in the way of matching a request to the desired response One way Stub-O-Matic can solvethis problem is by the use of lsquomatchersrsquo Matchers are parts of the request that matter to you only what is needed tofind the correct response file
For example a request may include
ltsessionidgt123456ltsessionidgtlttransactionidgt0987654321-9999lttransactionidgt
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Of these 4 lines only the last 2 matter the first 2 lines will differ from one request to the next and be set by the systemmaking the request This request then will be different each time it is made How can one easily match a request likethis to a canned response file without writing code for each requestresponse pair The solution used by Stub-O-Maticis to not store requestresponse file pairs but rather store matchers and the response In the case above the matcherswould be
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Stub-O-Matic contains code which will take a request as input search for the response with the best matchers Everymatcher must be contained in the request to be a match When a match is found the corresponding response is servedback
All matchers must be present in the request to make a match Stub-O-Matic will return the response from the firstmatch if there is more than one possible match Note that whitespace and carriage returns are ignored when matching
Alternatives to removing parts of the request you donrsquot want to match on are
1 using a matcher template to substitute ids or dates etc from the request into the matcher OR
2 using a user exit to run custom code A convenience class XMLManglerExit is provided for XML payloads toignore XMl elements andor attributes from the matching process
74 Templated Matcher
This is an example of a template matcher than extracts values from the request into the matcher The xmltree object isan lxml parsed root element passed into the template
ltrequestgtlttransidgtxmltreexpath(requesttransid)[0]textlttransidgtltdtgt2009-10-24T000000+0000ltdtgtltflightnumbergt0455ltflightnumbergtltrequestgt
75 Templated Responses
A hand crafted Stubo response may contain variables which are substituted in For example a response may contain
set flight_nbr=XX1234 The flight number is flight_nbr
18 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
The resulting response served wil be
The flight number is XX1234
Code may be used for example adding random elements to a response
import random set rnd_nbr = randomrandint(1200) Hello World 1000 + rnd_nbr
The resulting response served will be
Hello World - followed by a number between 1000 and 1200
To put todayrsquos date in a response include the following
today_strformat(dmy)
Stubo responses are run through a Tornado template and any logic or commands allowed in these templates can beused See httpwwwtornadoweborgenstabletemplatehtml for details
76 Request Data in Responses
761 Text Manipulation
It is possible to pull data from the stub request and put it in the response Say you have 200 stubs that are identicalapart from a userid They could be condensed into one stub which uses the userid from the request in the responserequest
ltuseridgtabc123ltuseridgt
matcher
ltuseridgt
response template (using Python)
Hello to request_text[814]
response
Hello to abc123
762 XML Manipulation
Sometimes it is easier to use XPATH to pull data from a request and place it in a response If you need the SessionIdfrom this request
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtJohn1234ltwbsSessionIdgt
Pull the SessionId from the request and use it within the response template
76 Request Data in Responses 19
Stub-O-Matic Documentation Release 0X beta
set namespaces = soapenv httpschemasxmlsoaporgsoapenvelopewbs httpxmlaaacomws200901WBS_Session-20xsd
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtxmltreexpath(soapenvHeaderwbsSessionwbsSessionId
namespaces=namespaces)[0]textltwbsSessionIdgt
Note the xmltree variable is the parsed xml request (as an lxml Root object) made available to the template if therequest is valid xml
Another example - no namespaces in the request Request
ltxml version=10 encoding=UTF-8gtltCompensateCustomersCheck APCWCustom=-1 LocalCurrencyCode=GBP NumberOfAdults=1gt
Response
set currency_code = xmltreexpath(CompensateCustomersCheck)[0]attrib[LocalCurrencyCode] ltresponse LocalCurrencyCode=currency_codegtpay meltresponsegt
77 Stateful Stubs
State is important in simulating back-end systems For example if one was to check-in passenger Bob on a particularflight the back-end would allow this request to succeed only one time and subsequent attempts to check-in the samepassenger on the same flight would be rejected Stubo can emulate this functionality or if desired for repeatable testsStubo can allow Bob to be checked in multiple times on the same flight
Stateful stubbing is enabled automatically If stubs are recorded or loaded in which the same request returns differentresponses Stubo will remember this and internally keep track of which response should come next
If your test is to check-in Bob multiple times simply do not load the stubs representing the check-in rejection andStubo will allow multiple check-ins
78 User Exits
Stub-O-Matic provides hooks into the runtime to execute custom user code Stubo can execute your custom code totransform stub matchers requests andor responses This is a powerful mechanism when your stubs contain dynamicdata or need to change over time This can be useful to perform lsquointelligent stubbingrsquo for example to roll dates Theuser exit API is contained in the stuboextuser_exit module
781 API
782 XML Auto Mangling
If your data is XML a convenience class XMLUserExit is provided to enable auto mangling
20 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
783 API
79 Hooks
710 Modules
Modules are added with the putmodule command and referenced via the ext_module url arg in the putstub call Forexample
putmodulename=staticcmdstestsextxsltmanglerpydeletestubsscenario=mangler_xsltbeginsessionscenario=mangler_xsltampsession=mangler_xslt1ampmode=recordputstubsession=mangler_xslt1ampext_module=mangler firstrequest firstresponseendsessionsession=mangler_xslt1
This example dynamically imports a test mangler from the Stubo server Typically these external python modules thathook into the Stubo runtime would be sourced from source control system using a URL
Note user exit processing is only enabled if the lsquoext_modulersquo variable is set to a pre-loaded module during the putstubcall Each Stub-O-Matic host URL has its own copy of any loaded modules Such modules are available to any testand as such should typically be loaded once per test suite
711 Splitting
Useful to removing random elements from matchers It can happen that the system under test will include seeminglyrandom elements in the request This will cause the next run of the test to not match This problem can be solved byprocessing matchers when loading - during a record A matcher can be split into two with the variable text left outFuture request must match both of the matchers to get the response An example is
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N11111234567806P1ltCommandgtltCryptic_GetScreen_Querygt
Which becomes two matchers
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N1111
and
06P1ltCommandgtltCryptic_GetScreen_Querygt
cutting out the random number lsquo12345678rsquo in the middle
This matcher splitting can be done manually or automatically with a user exit Seestaticcmdstestsextsplitters1firstcommands for and example It uses the rules module at staticcmdstestsextaaapy
Multiple matchers excluding random request elements can be created manually and loaded as such
putstubsession=splitter_record splitter_1textMatcher splitter_2textMatcher splitterresponse
712 Caching Values
If emulating back-end behaviour means writing some code for a particularly tricky behaviour Stubo exposes a simplekey-value cache API to matchers and responses
79 Hooks 21
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
CHAPTER 5
GUI
51 Tracker
The Stubo lsquoTrackerrsquo page shows details of all requests submitted to Stub-O-Matic and the received response Clickon the link in the function column to show a detailed view Debug level tracking can be added to a getresponse callif lsquotracking_level=fullrsquo is added to the call Matching errors will also trigger some extra debug data to be added to anormal track record
Stubo can not keep tracker data forever so it will eventually drop off the server as space is used up
The tracker page shows all Stubo actions performed on the server Filters are provided to filter on session datetimeresponse times and errors
To go into detail on a particular transaction click the link in the lsquoFunctionrsquo column or lsquoDetailsrsquo in the lsquoStubo Responsersquocolumn For a getresponse call the matchers and responses for the call can be found from the gtgtgt view scenario details(stub matchers and responses) link at the top of the details page
Filtering The tracker filter fields should be self explanatory By default the tracker page shows only data sent to theStubo URL shown at the top of the browser To see tracker items from other URLs on the same hardware tick thelsquoAll Hostsrsquo box
52 Manage
The manage page allows users to execute command files and view system state
The list of loaded stubs includes the date the stubs were loaded into Stubo and the last time the stub was used Clickingthe scenario name shows the stored matchers and responses for the scenario
Also available is a command file scratchpad for entering one or more commands For example one could enter
endsessionsession=first_sessiondeletestubsscenario=firstendsessionsession=second_sessiondeletestubsscenario=second
All the above commands would then be executed See the results on the tracker page
The manage page also shows stubs delay policies and any external modules loaded
13
Stub-O-Matic Documentation Release 0X beta
53 Analytics
The lsquoAnalyticsrsquo tab takes you to a dashboard showing statistics about the Stubo server The orange bar along the topallows you to select different Stubo servers and time periods The first of the 4 squares on the right turns on a legend
Charts
chart title descriptionstubssecondresponse rate
number of stubs requested per second
delay added user defined delays to stubs these can be specified to emulate response times of thesystem being stubbed
latency average time taken to return a stub excluding network and user defined delaysrequestresponse size size in kb of the requests to stubo and responses deliveredhost response latency where Stubo is deployed across multiple servers this chart compares the different servers
against each otherfailure rate percent of failed requests (HTTP response gt= 400)health checks Stubo has a regular health check ping to demonstrate that all systems are working
properlyrunning putstubcount
cumulative total of stubs loaded
running responsecount
cumulative total of stubs retrieved
Also note the Stubo API documentation for getstats
14 Chapter 5 GUI
CHAPTER 6
Integration with Test Tools
Stub-O-Matic works with all major software test execution tools Stubo of course does not run tests or evaluateresults it emulates back-end systems needed by tests
Data dependent software tests typically consist of 3 steps
1 Data setup
2 Test execution
3 Data teardown
Working with Stubo to provide back-end data is no different
Stuborsquos API can be used to setup or teardown data directly Stubo command files offer a shorthand for using the APIand are much more compact Any test tool that can make HTTP calls can use Stubo
61 Test steps example
1 Trigger an http call similar to this from the test tool of your choice
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_setupcommands
2 Run tests
3 Trigger a teardown http call similar to this from the test tool
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_teardowncommands
Setup would typically i) delete stubs ii) set any delays iii) submit a recording via a series of putstub calls and finallyiv) begin a session in playback
Teardown would typically i) end session ii) delete stubs
Setup and teardown can be performed with a mixture of command files (YAML or commands) and direct HTTP APIcalls to stubo
15
Stub-O-Matic Documentation Release 0X beta
16 Chapter 6 Integration with Test Tools
CHAPTER 7
Stubbing
71 Add Delay to a Response
To emulate response times of systems being stubbed one can set a delay on stubs Delays are set on the stub responsenot on the request to stubo This is because one request into the system being tested can result in several back-endstubocalls each of which might need different delay settings
Stubo provides reusable delay_policy objects which can be used across many stubs and across sessions The de-lay_policy can be altered on the fly during a loadperformance test if desired There are currently two delay_policytypes to choose from (1) fixed and (2)random with mean and standard deviation
A fixed delay policy can be created or modified with a commnand such as
putdelay_policyname=a_fixed_delay_policyampdelay_type=fixedampmilliseconds=200
Note that the value for the lsquonamersquo can be set to any string and is used again in the putstub A variable delay can be setup as follows
putdelay_policyname=a_normalvariate_delay_policyampdelay_type=normalvariateampmean=100ampstddev=50
Allowed lsquodelay typersquo values are lsquofixedrsquo and lsquonormalvariatersquo The next step is to use the delay policy when loadingstubs eg
putstubsession=abcampdelay_policy=a_fixed_delay_policy
bull Whenever the stub above is used the named delay policy will be applied
bull Delay policies may be altered on the fly during a test run if desired To change a delay on the fly simply repeatthe putdelay_policy command with a different value Typically this would be done directly by the test toolrather than from a command file
bull Delay policies can be used across multiple sessions This allows for easier alteration of policies where manysessions use the same policy It also means that you should name policies carefully and only alter or sharepolicies you are in control of
72 Matching
Stub-O-Matic currently supports various types of matchers
17
Stub-O-Matic Documentation Release 0X beta
73 Body contains matching
One or more matchers can be defined whereby all matchers must be contained in the request to return a specifiedresponse A typical difficulty with system requests is that they often contain superfluous information such as sessionIDs or time stamps that get in the way of matching a request to the desired response One way Stub-O-Matic can solvethis problem is by the use of lsquomatchersrsquo Matchers are parts of the request that matter to you only what is needed tofind the correct response file
For example a request may include
ltsessionidgt123456ltsessionidgtlttransactionidgt0987654321-9999lttransactionidgt
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Of these 4 lines only the last 2 matter the first 2 lines will differ from one request to the next and be set by the systemmaking the request This request then will be different each time it is made How can one easily match a request likethis to a canned response file without writing code for each requestresponse pair The solution used by Stub-O-Maticis to not store requestresponse file pairs but rather store matchers and the response In the case above the matcherswould be
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Stub-O-Matic contains code which will take a request as input search for the response with the best matchers Everymatcher must be contained in the request to be a match When a match is found the corresponding response is servedback
All matchers must be present in the request to make a match Stub-O-Matic will return the response from the firstmatch if there is more than one possible match Note that whitespace and carriage returns are ignored when matching
Alternatives to removing parts of the request you donrsquot want to match on are
1 using a matcher template to substitute ids or dates etc from the request into the matcher OR
2 using a user exit to run custom code A convenience class XMLManglerExit is provided for XML payloads toignore XMl elements andor attributes from the matching process
74 Templated Matcher
This is an example of a template matcher than extracts values from the request into the matcher The xmltree object isan lxml parsed root element passed into the template
ltrequestgtlttransidgtxmltreexpath(requesttransid)[0]textlttransidgtltdtgt2009-10-24T000000+0000ltdtgtltflightnumbergt0455ltflightnumbergtltrequestgt
75 Templated Responses
A hand crafted Stubo response may contain variables which are substituted in For example a response may contain
set flight_nbr=XX1234 The flight number is flight_nbr
18 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
The resulting response served wil be
The flight number is XX1234
Code may be used for example adding random elements to a response
import random set rnd_nbr = randomrandint(1200) Hello World 1000 + rnd_nbr
The resulting response served will be
Hello World - followed by a number between 1000 and 1200
To put todayrsquos date in a response include the following
today_strformat(dmy)
Stubo responses are run through a Tornado template and any logic or commands allowed in these templates can beused See httpwwwtornadoweborgenstabletemplatehtml for details
76 Request Data in Responses
761 Text Manipulation
It is possible to pull data from the stub request and put it in the response Say you have 200 stubs that are identicalapart from a userid They could be condensed into one stub which uses the userid from the request in the responserequest
ltuseridgtabc123ltuseridgt
matcher
ltuseridgt
response template (using Python)
Hello to request_text[814]
response
Hello to abc123
762 XML Manipulation
Sometimes it is easier to use XPATH to pull data from a request and place it in a response If you need the SessionIdfrom this request
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtJohn1234ltwbsSessionIdgt
Pull the SessionId from the request and use it within the response template
76 Request Data in Responses 19
Stub-O-Matic Documentation Release 0X beta
set namespaces = soapenv httpschemasxmlsoaporgsoapenvelopewbs httpxmlaaacomws200901WBS_Session-20xsd
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtxmltreexpath(soapenvHeaderwbsSessionwbsSessionId
namespaces=namespaces)[0]textltwbsSessionIdgt
Note the xmltree variable is the parsed xml request (as an lxml Root object) made available to the template if therequest is valid xml
Another example - no namespaces in the request Request
ltxml version=10 encoding=UTF-8gtltCompensateCustomersCheck APCWCustom=-1 LocalCurrencyCode=GBP NumberOfAdults=1gt
Response
set currency_code = xmltreexpath(CompensateCustomersCheck)[0]attrib[LocalCurrencyCode] ltresponse LocalCurrencyCode=currency_codegtpay meltresponsegt
77 Stateful Stubs
State is important in simulating back-end systems For example if one was to check-in passenger Bob on a particularflight the back-end would allow this request to succeed only one time and subsequent attempts to check-in the samepassenger on the same flight would be rejected Stubo can emulate this functionality or if desired for repeatable testsStubo can allow Bob to be checked in multiple times on the same flight
Stateful stubbing is enabled automatically If stubs are recorded or loaded in which the same request returns differentresponses Stubo will remember this and internally keep track of which response should come next
If your test is to check-in Bob multiple times simply do not load the stubs representing the check-in rejection andStubo will allow multiple check-ins
78 User Exits
Stub-O-Matic provides hooks into the runtime to execute custom user code Stubo can execute your custom code totransform stub matchers requests andor responses This is a powerful mechanism when your stubs contain dynamicdata or need to change over time This can be useful to perform lsquointelligent stubbingrsquo for example to roll dates Theuser exit API is contained in the stuboextuser_exit module
781 API
782 XML Auto Mangling
If your data is XML a convenience class XMLUserExit is provided to enable auto mangling
20 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
783 API
79 Hooks
710 Modules
Modules are added with the putmodule command and referenced via the ext_module url arg in the putstub call Forexample
putmodulename=staticcmdstestsextxsltmanglerpydeletestubsscenario=mangler_xsltbeginsessionscenario=mangler_xsltampsession=mangler_xslt1ampmode=recordputstubsession=mangler_xslt1ampext_module=mangler firstrequest firstresponseendsessionsession=mangler_xslt1
This example dynamically imports a test mangler from the Stubo server Typically these external python modules thathook into the Stubo runtime would be sourced from source control system using a URL
Note user exit processing is only enabled if the lsquoext_modulersquo variable is set to a pre-loaded module during the putstubcall Each Stub-O-Matic host URL has its own copy of any loaded modules Such modules are available to any testand as such should typically be loaded once per test suite
711 Splitting
Useful to removing random elements from matchers It can happen that the system under test will include seeminglyrandom elements in the request This will cause the next run of the test to not match This problem can be solved byprocessing matchers when loading - during a record A matcher can be split into two with the variable text left outFuture request must match both of the matchers to get the response An example is
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N11111234567806P1ltCommandgtltCryptic_GetScreen_Querygt
Which becomes two matchers
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N1111
and
06P1ltCommandgtltCryptic_GetScreen_Querygt
cutting out the random number lsquo12345678rsquo in the middle
This matcher splitting can be done manually or automatically with a user exit Seestaticcmdstestsextsplitters1firstcommands for and example It uses the rules module at staticcmdstestsextaaapy
Multiple matchers excluding random request elements can be created manually and loaded as such
putstubsession=splitter_record splitter_1textMatcher splitter_2textMatcher splitterresponse
712 Caching Values
If emulating back-end behaviour means writing some code for a particularly tricky behaviour Stubo exposes a simplekey-value cache API to matchers and responses
79 Hooks 21
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
Stub-O-Matic Documentation Release 0X beta
53 Analytics
The lsquoAnalyticsrsquo tab takes you to a dashboard showing statistics about the Stubo server The orange bar along the topallows you to select different Stubo servers and time periods The first of the 4 squares on the right turns on a legend
Charts
chart title descriptionstubssecondresponse rate
number of stubs requested per second
delay added user defined delays to stubs these can be specified to emulate response times of thesystem being stubbed
latency average time taken to return a stub excluding network and user defined delaysrequestresponse size size in kb of the requests to stubo and responses deliveredhost response latency where Stubo is deployed across multiple servers this chart compares the different servers
against each otherfailure rate percent of failed requests (HTTP response gt= 400)health checks Stubo has a regular health check ping to demonstrate that all systems are working
properlyrunning putstubcount
cumulative total of stubs loaded
running responsecount
cumulative total of stubs retrieved
Also note the Stubo API documentation for getstats
14 Chapter 5 GUI
CHAPTER 6
Integration with Test Tools
Stub-O-Matic works with all major software test execution tools Stubo of course does not run tests or evaluateresults it emulates back-end systems needed by tests
Data dependent software tests typically consist of 3 steps
1 Data setup
2 Test execution
3 Data teardown
Working with Stubo to provide back-end data is no different
Stuborsquos API can be used to setup or teardown data directly Stubo command files offer a shorthand for using the APIand are much more compact Any test tool that can make HTTP calls can use Stubo
61 Test steps example
1 Trigger an http call similar to this from the test tool of your choice
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_setupcommands
2 Run tests
3 Trigger a teardown http call similar to this from the test tool
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_teardowncommands
Setup would typically i) delete stubs ii) set any delays iii) submit a recording via a series of putstub calls and finallyiv) begin a session in playback
Teardown would typically i) end session ii) delete stubs
Setup and teardown can be performed with a mixture of command files (YAML or commands) and direct HTTP APIcalls to stubo
15
Stub-O-Matic Documentation Release 0X beta
16 Chapter 6 Integration with Test Tools
CHAPTER 7
Stubbing
71 Add Delay to a Response
To emulate response times of systems being stubbed one can set a delay on stubs Delays are set on the stub responsenot on the request to stubo This is because one request into the system being tested can result in several back-endstubocalls each of which might need different delay settings
Stubo provides reusable delay_policy objects which can be used across many stubs and across sessions The de-lay_policy can be altered on the fly during a loadperformance test if desired There are currently two delay_policytypes to choose from (1) fixed and (2)random with mean and standard deviation
A fixed delay policy can be created or modified with a commnand such as
putdelay_policyname=a_fixed_delay_policyampdelay_type=fixedampmilliseconds=200
Note that the value for the lsquonamersquo can be set to any string and is used again in the putstub A variable delay can be setup as follows
putdelay_policyname=a_normalvariate_delay_policyampdelay_type=normalvariateampmean=100ampstddev=50
Allowed lsquodelay typersquo values are lsquofixedrsquo and lsquonormalvariatersquo The next step is to use the delay policy when loadingstubs eg
putstubsession=abcampdelay_policy=a_fixed_delay_policy
bull Whenever the stub above is used the named delay policy will be applied
bull Delay policies may be altered on the fly during a test run if desired To change a delay on the fly simply repeatthe putdelay_policy command with a different value Typically this would be done directly by the test toolrather than from a command file
bull Delay policies can be used across multiple sessions This allows for easier alteration of policies where manysessions use the same policy It also means that you should name policies carefully and only alter or sharepolicies you are in control of
72 Matching
Stub-O-Matic currently supports various types of matchers
17
Stub-O-Matic Documentation Release 0X beta
73 Body contains matching
One or more matchers can be defined whereby all matchers must be contained in the request to return a specifiedresponse A typical difficulty with system requests is that they often contain superfluous information such as sessionIDs or time stamps that get in the way of matching a request to the desired response One way Stub-O-Matic can solvethis problem is by the use of lsquomatchersrsquo Matchers are parts of the request that matter to you only what is needed tofind the correct response file
For example a request may include
ltsessionidgt123456ltsessionidgtlttransactionidgt0987654321-9999lttransactionidgt
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Of these 4 lines only the last 2 matter the first 2 lines will differ from one request to the next and be set by the systemmaking the request This request then will be different each time it is made How can one easily match a request likethis to a canned response file without writing code for each requestresponse pair The solution used by Stub-O-Maticis to not store requestresponse file pairs but rather store matchers and the response In the case above the matcherswould be
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Stub-O-Matic contains code which will take a request as input search for the response with the best matchers Everymatcher must be contained in the request to be a match When a match is found the corresponding response is servedback
All matchers must be present in the request to make a match Stub-O-Matic will return the response from the firstmatch if there is more than one possible match Note that whitespace and carriage returns are ignored when matching
Alternatives to removing parts of the request you donrsquot want to match on are
1 using a matcher template to substitute ids or dates etc from the request into the matcher OR
2 using a user exit to run custom code A convenience class XMLManglerExit is provided for XML payloads toignore XMl elements andor attributes from the matching process
74 Templated Matcher
This is an example of a template matcher than extracts values from the request into the matcher The xmltree object isan lxml parsed root element passed into the template
ltrequestgtlttransidgtxmltreexpath(requesttransid)[0]textlttransidgtltdtgt2009-10-24T000000+0000ltdtgtltflightnumbergt0455ltflightnumbergtltrequestgt
75 Templated Responses
A hand crafted Stubo response may contain variables which are substituted in For example a response may contain
set flight_nbr=XX1234 The flight number is flight_nbr
18 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
The resulting response served wil be
The flight number is XX1234
Code may be used for example adding random elements to a response
import random set rnd_nbr = randomrandint(1200) Hello World 1000 + rnd_nbr
The resulting response served will be
Hello World - followed by a number between 1000 and 1200
To put todayrsquos date in a response include the following
today_strformat(dmy)
Stubo responses are run through a Tornado template and any logic or commands allowed in these templates can beused See httpwwwtornadoweborgenstabletemplatehtml for details
76 Request Data in Responses
761 Text Manipulation
It is possible to pull data from the stub request and put it in the response Say you have 200 stubs that are identicalapart from a userid They could be condensed into one stub which uses the userid from the request in the responserequest
ltuseridgtabc123ltuseridgt
matcher
ltuseridgt
response template (using Python)
Hello to request_text[814]
response
Hello to abc123
762 XML Manipulation
Sometimes it is easier to use XPATH to pull data from a request and place it in a response If you need the SessionIdfrom this request
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtJohn1234ltwbsSessionIdgt
Pull the SessionId from the request and use it within the response template
76 Request Data in Responses 19
Stub-O-Matic Documentation Release 0X beta
set namespaces = soapenv httpschemasxmlsoaporgsoapenvelopewbs httpxmlaaacomws200901WBS_Session-20xsd
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtxmltreexpath(soapenvHeaderwbsSessionwbsSessionId
namespaces=namespaces)[0]textltwbsSessionIdgt
Note the xmltree variable is the parsed xml request (as an lxml Root object) made available to the template if therequest is valid xml
Another example - no namespaces in the request Request
ltxml version=10 encoding=UTF-8gtltCompensateCustomersCheck APCWCustom=-1 LocalCurrencyCode=GBP NumberOfAdults=1gt
Response
set currency_code = xmltreexpath(CompensateCustomersCheck)[0]attrib[LocalCurrencyCode] ltresponse LocalCurrencyCode=currency_codegtpay meltresponsegt
77 Stateful Stubs
State is important in simulating back-end systems For example if one was to check-in passenger Bob on a particularflight the back-end would allow this request to succeed only one time and subsequent attempts to check-in the samepassenger on the same flight would be rejected Stubo can emulate this functionality or if desired for repeatable testsStubo can allow Bob to be checked in multiple times on the same flight
Stateful stubbing is enabled automatically If stubs are recorded or loaded in which the same request returns differentresponses Stubo will remember this and internally keep track of which response should come next
If your test is to check-in Bob multiple times simply do not load the stubs representing the check-in rejection andStubo will allow multiple check-ins
78 User Exits
Stub-O-Matic provides hooks into the runtime to execute custom user code Stubo can execute your custom code totransform stub matchers requests andor responses This is a powerful mechanism when your stubs contain dynamicdata or need to change over time This can be useful to perform lsquointelligent stubbingrsquo for example to roll dates Theuser exit API is contained in the stuboextuser_exit module
781 API
782 XML Auto Mangling
If your data is XML a convenience class XMLUserExit is provided to enable auto mangling
20 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
783 API
79 Hooks
710 Modules
Modules are added with the putmodule command and referenced via the ext_module url arg in the putstub call Forexample
putmodulename=staticcmdstestsextxsltmanglerpydeletestubsscenario=mangler_xsltbeginsessionscenario=mangler_xsltampsession=mangler_xslt1ampmode=recordputstubsession=mangler_xslt1ampext_module=mangler firstrequest firstresponseendsessionsession=mangler_xslt1
This example dynamically imports a test mangler from the Stubo server Typically these external python modules thathook into the Stubo runtime would be sourced from source control system using a URL
Note user exit processing is only enabled if the lsquoext_modulersquo variable is set to a pre-loaded module during the putstubcall Each Stub-O-Matic host URL has its own copy of any loaded modules Such modules are available to any testand as such should typically be loaded once per test suite
711 Splitting
Useful to removing random elements from matchers It can happen that the system under test will include seeminglyrandom elements in the request This will cause the next run of the test to not match This problem can be solved byprocessing matchers when loading - during a record A matcher can be split into two with the variable text left outFuture request must match both of the matchers to get the response An example is
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N11111234567806P1ltCommandgtltCryptic_GetScreen_Querygt
Which becomes two matchers
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N1111
and
06P1ltCommandgtltCryptic_GetScreen_Querygt
cutting out the random number lsquo12345678rsquo in the middle
This matcher splitting can be done manually or automatically with a user exit Seestaticcmdstestsextsplitters1firstcommands for and example It uses the rules module at staticcmdstestsextaaapy
Multiple matchers excluding random request elements can be created manually and loaded as such
putstubsession=splitter_record splitter_1textMatcher splitter_2textMatcher splitterresponse
712 Caching Values
If emulating back-end behaviour means writing some code for a particularly tricky behaviour Stubo exposes a simplekey-value cache API to matchers and responses
79 Hooks 21
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
CHAPTER 6
Integration with Test Tools
Stub-O-Matic works with all major software test execution tools Stubo of course does not run tests or evaluateresults it emulates back-end systems needed by tests
Data dependent software tests typically consist of 3 steps
1 Data setup
2 Test execution
3 Data teardown
Working with Stubo to provide back-end data is no different
Stuborsquos API can be used to setup or teardown data directly Stubo command files offer a shorthand for using the APIand are much more compact Any test tool that can make HTTP calls can use Stubo
61 Test steps example
1 Trigger an http call similar to this from the test tool of your choice
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_setupcommands
2 Run tests
3 Trigger a teardown http call similar to this from the test tool
httpmystubocomstuboapiexeccmdscmdfile=httpmy_stubscheckin_teardowncommands
Setup would typically i) delete stubs ii) set any delays iii) submit a recording via a series of putstub calls and finallyiv) begin a session in playback
Teardown would typically i) end session ii) delete stubs
Setup and teardown can be performed with a mixture of command files (YAML or commands) and direct HTTP APIcalls to stubo
15
Stub-O-Matic Documentation Release 0X beta
16 Chapter 6 Integration with Test Tools
CHAPTER 7
Stubbing
71 Add Delay to a Response
To emulate response times of systems being stubbed one can set a delay on stubs Delays are set on the stub responsenot on the request to stubo This is because one request into the system being tested can result in several back-endstubocalls each of which might need different delay settings
Stubo provides reusable delay_policy objects which can be used across many stubs and across sessions The de-lay_policy can be altered on the fly during a loadperformance test if desired There are currently two delay_policytypes to choose from (1) fixed and (2)random with mean and standard deviation
A fixed delay policy can be created or modified with a commnand such as
putdelay_policyname=a_fixed_delay_policyampdelay_type=fixedampmilliseconds=200
Note that the value for the lsquonamersquo can be set to any string and is used again in the putstub A variable delay can be setup as follows
putdelay_policyname=a_normalvariate_delay_policyampdelay_type=normalvariateampmean=100ampstddev=50
Allowed lsquodelay typersquo values are lsquofixedrsquo and lsquonormalvariatersquo The next step is to use the delay policy when loadingstubs eg
putstubsession=abcampdelay_policy=a_fixed_delay_policy
bull Whenever the stub above is used the named delay policy will be applied
bull Delay policies may be altered on the fly during a test run if desired To change a delay on the fly simply repeatthe putdelay_policy command with a different value Typically this would be done directly by the test toolrather than from a command file
bull Delay policies can be used across multiple sessions This allows for easier alteration of policies where manysessions use the same policy It also means that you should name policies carefully and only alter or sharepolicies you are in control of
72 Matching
Stub-O-Matic currently supports various types of matchers
17
Stub-O-Matic Documentation Release 0X beta
73 Body contains matching
One or more matchers can be defined whereby all matchers must be contained in the request to return a specifiedresponse A typical difficulty with system requests is that they often contain superfluous information such as sessionIDs or time stamps that get in the way of matching a request to the desired response One way Stub-O-Matic can solvethis problem is by the use of lsquomatchersrsquo Matchers are parts of the request that matter to you only what is needed tofind the correct response file
For example a request may include
ltsessionidgt123456ltsessionidgtlttransactionidgt0987654321-9999lttransactionidgt
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Of these 4 lines only the last 2 matter the first 2 lines will differ from one request to the next and be set by the systemmaking the request This request then will be different each time it is made How can one easily match a request likethis to a canned response file without writing code for each requestresponse pair The solution used by Stub-O-Maticis to not store requestresponse file pairs but rather store matchers and the response In the case above the matcherswould be
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Stub-O-Matic contains code which will take a request as input search for the response with the best matchers Everymatcher must be contained in the request to be a match When a match is found the corresponding response is servedback
All matchers must be present in the request to make a match Stub-O-Matic will return the response from the firstmatch if there is more than one possible match Note that whitespace and carriage returns are ignored when matching
Alternatives to removing parts of the request you donrsquot want to match on are
1 using a matcher template to substitute ids or dates etc from the request into the matcher OR
2 using a user exit to run custom code A convenience class XMLManglerExit is provided for XML payloads toignore XMl elements andor attributes from the matching process
74 Templated Matcher
This is an example of a template matcher than extracts values from the request into the matcher The xmltree object isan lxml parsed root element passed into the template
ltrequestgtlttransidgtxmltreexpath(requesttransid)[0]textlttransidgtltdtgt2009-10-24T000000+0000ltdtgtltflightnumbergt0455ltflightnumbergtltrequestgt
75 Templated Responses
A hand crafted Stubo response may contain variables which are substituted in For example a response may contain
set flight_nbr=XX1234 The flight number is flight_nbr
18 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
The resulting response served wil be
The flight number is XX1234
Code may be used for example adding random elements to a response
import random set rnd_nbr = randomrandint(1200) Hello World 1000 + rnd_nbr
The resulting response served will be
Hello World - followed by a number between 1000 and 1200
To put todayrsquos date in a response include the following
today_strformat(dmy)
Stubo responses are run through a Tornado template and any logic or commands allowed in these templates can beused See httpwwwtornadoweborgenstabletemplatehtml for details
76 Request Data in Responses
761 Text Manipulation
It is possible to pull data from the stub request and put it in the response Say you have 200 stubs that are identicalapart from a userid They could be condensed into one stub which uses the userid from the request in the responserequest
ltuseridgtabc123ltuseridgt
matcher
ltuseridgt
response template (using Python)
Hello to request_text[814]
response
Hello to abc123
762 XML Manipulation
Sometimes it is easier to use XPATH to pull data from a request and place it in a response If you need the SessionIdfrom this request
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtJohn1234ltwbsSessionIdgt
Pull the SessionId from the request and use it within the response template
76 Request Data in Responses 19
Stub-O-Matic Documentation Release 0X beta
set namespaces = soapenv httpschemasxmlsoaporgsoapenvelopewbs httpxmlaaacomws200901WBS_Session-20xsd
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtxmltreexpath(soapenvHeaderwbsSessionwbsSessionId
namespaces=namespaces)[0]textltwbsSessionIdgt
Note the xmltree variable is the parsed xml request (as an lxml Root object) made available to the template if therequest is valid xml
Another example - no namespaces in the request Request
ltxml version=10 encoding=UTF-8gtltCompensateCustomersCheck APCWCustom=-1 LocalCurrencyCode=GBP NumberOfAdults=1gt
Response
set currency_code = xmltreexpath(CompensateCustomersCheck)[0]attrib[LocalCurrencyCode] ltresponse LocalCurrencyCode=currency_codegtpay meltresponsegt
77 Stateful Stubs
State is important in simulating back-end systems For example if one was to check-in passenger Bob on a particularflight the back-end would allow this request to succeed only one time and subsequent attempts to check-in the samepassenger on the same flight would be rejected Stubo can emulate this functionality or if desired for repeatable testsStubo can allow Bob to be checked in multiple times on the same flight
Stateful stubbing is enabled automatically If stubs are recorded or loaded in which the same request returns differentresponses Stubo will remember this and internally keep track of which response should come next
If your test is to check-in Bob multiple times simply do not load the stubs representing the check-in rejection andStubo will allow multiple check-ins
78 User Exits
Stub-O-Matic provides hooks into the runtime to execute custom user code Stubo can execute your custom code totransform stub matchers requests andor responses This is a powerful mechanism when your stubs contain dynamicdata or need to change over time This can be useful to perform lsquointelligent stubbingrsquo for example to roll dates Theuser exit API is contained in the stuboextuser_exit module
781 API
782 XML Auto Mangling
If your data is XML a convenience class XMLUserExit is provided to enable auto mangling
20 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
783 API
79 Hooks
710 Modules
Modules are added with the putmodule command and referenced via the ext_module url arg in the putstub call Forexample
putmodulename=staticcmdstestsextxsltmanglerpydeletestubsscenario=mangler_xsltbeginsessionscenario=mangler_xsltampsession=mangler_xslt1ampmode=recordputstubsession=mangler_xslt1ampext_module=mangler firstrequest firstresponseendsessionsession=mangler_xslt1
This example dynamically imports a test mangler from the Stubo server Typically these external python modules thathook into the Stubo runtime would be sourced from source control system using a URL
Note user exit processing is only enabled if the lsquoext_modulersquo variable is set to a pre-loaded module during the putstubcall Each Stub-O-Matic host URL has its own copy of any loaded modules Such modules are available to any testand as such should typically be loaded once per test suite
711 Splitting
Useful to removing random elements from matchers It can happen that the system under test will include seeminglyrandom elements in the request This will cause the next run of the test to not match This problem can be solved byprocessing matchers when loading - during a record A matcher can be split into two with the variable text left outFuture request must match both of the matchers to get the response An example is
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N11111234567806P1ltCommandgtltCryptic_GetScreen_Querygt
Which becomes two matchers
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N1111
and
06P1ltCommandgtltCryptic_GetScreen_Querygt
cutting out the random number lsquo12345678rsquo in the middle
This matcher splitting can be done manually or automatically with a user exit Seestaticcmdstestsextsplitters1firstcommands for and example It uses the rules module at staticcmdstestsextaaapy
Multiple matchers excluding random request elements can be created manually and loaded as such
putstubsession=splitter_record splitter_1textMatcher splitter_2textMatcher splitterresponse
712 Caching Values
If emulating back-end behaviour means writing some code for a particularly tricky behaviour Stubo exposes a simplekey-value cache API to matchers and responses
79 Hooks 21
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
Stub-O-Matic Documentation Release 0X beta
16 Chapter 6 Integration with Test Tools
CHAPTER 7
Stubbing
71 Add Delay to a Response
To emulate response times of systems being stubbed one can set a delay on stubs Delays are set on the stub responsenot on the request to stubo This is because one request into the system being tested can result in several back-endstubocalls each of which might need different delay settings
Stubo provides reusable delay_policy objects which can be used across many stubs and across sessions The de-lay_policy can be altered on the fly during a loadperformance test if desired There are currently two delay_policytypes to choose from (1) fixed and (2)random with mean and standard deviation
A fixed delay policy can be created or modified with a commnand such as
putdelay_policyname=a_fixed_delay_policyampdelay_type=fixedampmilliseconds=200
Note that the value for the lsquonamersquo can be set to any string and is used again in the putstub A variable delay can be setup as follows
putdelay_policyname=a_normalvariate_delay_policyampdelay_type=normalvariateampmean=100ampstddev=50
Allowed lsquodelay typersquo values are lsquofixedrsquo and lsquonormalvariatersquo The next step is to use the delay policy when loadingstubs eg
putstubsession=abcampdelay_policy=a_fixed_delay_policy
bull Whenever the stub above is used the named delay policy will be applied
bull Delay policies may be altered on the fly during a test run if desired To change a delay on the fly simply repeatthe putdelay_policy command with a different value Typically this would be done directly by the test toolrather than from a command file
bull Delay policies can be used across multiple sessions This allows for easier alteration of policies where manysessions use the same policy It also means that you should name policies carefully and only alter or sharepolicies you are in control of
72 Matching
Stub-O-Matic currently supports various types of matchers
17
Stub-O-Matic Documentation Release 0X beta
73 Body contains matching
One or more matchers can be defined whereby all matchers must be contained in the request to return a specifiedresponse A typical difficulty with system requests is that they often contain superfluous information such as sessionIDs or time stamps that get in the way of matching a request to the desired response One way Stub-O-Matic can solvethis problem is by the use of lsquomatchersrsquo Matchers are parts of the request that matter to you only what is needed tofind the correct response file
For example a request may include
ltsessionidgt123456ltsessionidgtlttransactionidgt0987654321-9999lttransactionidgt
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Of these 4 lines only the last 2 matter the first 2 lines will differ from one request to the next and be set by the systemmaking the request This request then will be different each time it is made How can one easily match a request likethis to a canned response file without writing code for each requestresponse pair The solution used by Stub-O-Maticis to not store requestresponse file pairs but rather store matchers and the response In the case above the matcherswould be
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Stub-O-Matic contains code which will take a request as input search for the response with the best matchers Everymatcher must be contained in the request to be a match When a match is found the corresponding response is servedback
All matchers must be present in the request to make a match Stub-O-Matic will return the response from the firstmatch if there is more than one possible match Note that whitespace and carriage returns are ignored when matching
Alternatives to removing parts of the request you donrsquot want to match on are
1 using a matcher template to substitute ids or dates etc from the request into the matcher OR
2 using a user exit to run custom code A convenience class XMLManglerExit is provided for XML payloads toignore XMl elements andor attributes from the matching process
74 Templated Matcher
This is an example of a template matcher than extracts values from the request into the matcher The xmltree object isan lxml parsed root element passed into the template
ltrequestgtlttransidgtxmltreexpath(requesttransid)[0]textlttransidgtltdtgt2009-10-24T000000+0000ltdtgtltflightnumbergt0455ltflightnumbergtltrequestgt
75 Templated Responses
A hand crafted Stubo response may contain variables which are substituted in For example a response may contain
set flight_nbr=XX1234 The flight number is flight_nbr
18 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
The resulting response served wil be
The flight number is XX1234
Code may be used for example adding random elements to a response
import random set rnd_nbr = randomrandint(1200) Hello World 1000 + rnd_nbr
The resulting response served will be
Hello World - followed by a number between 1000 and 1200
To put todayrsquos date in a response include the following
today_strformat(dmy)
Stubo responses are run through a Tornado template and any logic or commands allowed in these templates can beused See httpwwwtornadoweborgenstabletemplatehtml for details
76 Request Data in Responses
761 Text Manipulation
It is possible to pull data from the stub request and put it in the response Say you have 200 stubs that are identicalapart from a userid They could be condensed into one stub which uses the userid from the request in the responserequest
ltuseridgtabc123ltuseridgt
matcher
ltuseridgt
response template (using Python)
Hello to request_text[814]
response
Hello to abc123
762 XML Manipulation
Sometimes it is easier to use XPATH to pull data from a request and place it in a response If you need the SessionIdfrom this request
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtJohn1234ltwbsSessionIdgt
Pull the SessionId from the request and use it within the response template
76 Request Data in Responses 19
Stub-O-Matic Documentation Release 0X beta
set namespaces = soapenv httpschemasxmlsoaporgsoapenvelopewbs httpxmlaaacomws200901WBS_Session-20xsd
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtxmltreexpath(soapenvHeaderwbsSessionwbsSessionId
namespaces=namespaces)[0]textltwbsSessionIdgt
Note the xmltree variable is the parsed xml request (as an lxml Root object) made available to the template if therequest is valid xml
Another example - no namespaces in the request Request
ltxml version=10 encoding=UTF-8gtltCompensateCustomersCheck APCWCustom=-1 LocalCurrencyCode=GBP NumberOfAdults=1gt
Response
set currency_code = xmltreexpath(CompensateCustomersCheck)[0]attrib[LocalCurrencyCode] ltresponse LocalCurrencyCode=currency_codegtpay meltresponsegt
77 Stateful Stubs
State is important in simulating back-end systems For example if one was to check-in passenger Bob on a particularflight the back-end would allow this request to succeed only one time and subsequent attempts to check-in the samepassenger on the same flight would be rejected Stubo can emulate this functionality or if desired for repeatable testsStubo can allow Bob to be checked in multiple times on the same flight
Stateful stubbing is enabled automatically If stubs are recorded or loaded in which the same request returns differentresponses Stubo will remember this and internally keep track of which response should come next
If your test is to check-in Bob multiple times simply do not load the stubs representing the check-in rejection andStubo will allow multiple check-ins
78 User Exits
Stub-O-Matic provides hooks into the runtime to execute custom user code Stubo can execute your custom code totransform stub matchers requests andor responses This is a powerful mechanism when your stubs contain dynamicdata or need to change over time This can be useful to perform lsquointelligent stubbingrsquo for example to roll dates Theuser exit API is contained in the stuboextuser_exit module
781 API
782 XML Auto Mangling
If your data is XML a convenience class XMLUserExit is provided to enable auto mangling
20 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
783 API
79 Hooks
710 Modules
Modules are added with the putmodule command and referenced via the ext_module url arg in the putstub call Forexample
putmodulename=staticcmdstestsextxsltmanglerpydeletestubsscenario=mangler_xsltbeginsessionscenario=mangler_xsltampsession=mangler_xslt1ampmode=recordputstubsession=mangler_xslt1ampext_module=mangler firstrequest firstresponseendsessionsession=mangler_xslt1
This example dynamically imports a test mangler from the Stubo server Typically these external python modules thathook into the Stubo runtime would be sourced from source control system using a URL
Note user exit processing is only enabled if the lsquoext_modulersquo variable is set to a pre-loaded module during the putstubcall Each Stub-O-Matic host URL has its own copy of any loaded modules Such modules are available to any testand as such should typically be loaded once per test suite
711 Splitting
Useful to removing random elements from matchers It can happen that the system under test will include seeminglyrandom elements in the request This will cause the next run of the test to not match This problem can be solved byprocessing matchers when loading - during a record A matcher can be split into two with the variable text left outFuture request must match both of the matchers to get the response An example is
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N11111234567806P1ltCommandgtltCryptic_GetScreen_Querygt
Which becomes two matchers
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N1111
and
06P1ltCommandgtltCryptic_GetScreen_Querygt
cutting out the random number lsquo12345678rsquo in the middle
This matcher splitting can be done manually or automatically with a user exit Seestaticcmdstestsextsplitters1firstcommands for and example It uses the rules module at staticcmdstestsextaaapy
Multiple matchers excluding random request elements can be created manually and loaded as such
putstubsession=splitter_record splitter_1textMatcher splitter_2textMatcher splitterresponse
712 Caching Values
If emulating back-end behaviour means writing some code for a particularly tricky behaviour Stubo exposes a simplekey-value cache API to matchers and responses
79 Hooks 21
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
CHAPTER 7
Stubbing
71 Add Delay to a Response
To emulate response times of systems being stubbed one can set a delay on stubs Delays are set on the stub responsenot on the request to stubo This is because one request into the system being tested can result in several back-endstubocalls each of which might need different delay settings
Stubo provides reusable delay_policy objects which can be used across many stubs and across sessions The de-lay_policy can be altered on the fly during a loadperformance test if desired There are currently two delay_policytypes to choose from (1) fixed and (2)random with mean and standard deviation
A fixed delay policy can be created or modified with a commnand such as
putdelay_policyname=a_fixed_delay_policyampdelay_type=fixedampmilliseconds=200
Note that the value for the lsquonamersquo can be set to any string and is used again in the putstub A variable delay can be setup as follows
putdelay_policyname=a_normalvariate_delay_policyampdelay_type=normalvariateampmean=100ampstddev=50
Allowed lsquodelay typersquo values are lsquofixedrsquo and lsquonormalvariatersquo The next step is to use the delay policy when loadingstubs eg
putstubsession=abcampdelay_policy=a_fixed_delay_policy
bull Whenever the stub above is used the named delay policy will be applied
bull Delay policies may be altered on the fly during a test run if desired To change a delay on the fly simply repeatthe putdelay_policy command with a different value Typically this would be done directly by the test toolrather than from a command file
bull Delay policies can be used across multiple sessions This allows for easier alteration of policies where manysessions use the same policy It also means that you should name policies carefully and only alter or sharepolicies you are in control of
72 Matching
Stub-O-Matic currently supports various types of matchers
17
Stub-O-Matic Documentation Release 0X beta
73 Body contains matching
One or more matchers can be defined whereby all matchers must be contained in the request to return a specifiedresponse A typical difficulty with system requests is that they often contain superfluous information such as sessionIDs or time stamps that get in the way of matching a request to the desired response One way Stub-O-Matic can solvethis problem is by the use of lsquomatchersrsquo Matchers are parts of the request that matter to you only what is needed tofind the correct response file
For example a request may include
ltsessionidgt123456ltsessionidgtlttransactionidgt0987654321-9999lttransactionidgt
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Of these 4 lines only the last 2 matter the first 2 lines will differ from one request to the next and be set by the systemmaking the request This request then will be different each time it is made How can one easily match a request likethis to a canned response file without writing code for each requestresponse pair The solution used by Stub-O-Maticis to not store requestresponse file pairs but rather store matchers and the response In the case above the matcherswould be
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Stub-O-Matic contains code which will take a request as input search for the response with the best matchers Everymatcher must be contained in the request to be a match When a match is found the corresponding response is servedback
All matchers must be present in the request to make a match Stub-O-Matic will return the response from the firstmatch if there is more than one possible match Note that whitespace and carriage returns are ignored when matching
Alternatives to removing parts of the request you donrsquot want to match on are
1 using a matcher template to substitute ids or dates etc from the request into the matcher OR
2 using a user exit to run custom code A convenience class XMLManglerExit is provided for XML payloads toignore XMl elements andor attributes from the matching process
74 Templated Matcher
This is an example of a template matcher than extracts values from the request into the matcher The xmltree object isan lxml parsed root element passed into the template
ltrequestgtlttransidgtxmltreexpath(requesttransid)[0]textlttransidgtltdtgt2009-10-24T000000+0000ltdtgtltflightnumbergt0455ltflightnumbergtltrequestgt
75 Templated Responses
A hand crafted Stubo response may contain variables which are substituted in For example a response may contain
set flight_nbr=XX1234 The flight number is flight_nbr
18 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
The resulting response served wil be
The flight number is XX1234
Code may be used for example adding random elements to a response
import random set rnd_nbr = randomrandint(1200) Hello World 1000 + rnd_nbr
The resulting response served will be
Hello World - followed by a number between 1000 and 1200
To put todayrsquos date in a response include the following
today_strformat(dmy)
Stubo responses are run through a Tornado template and any logic or commands allowed in these templates can beused See httpwwwtornadoweborgenstabletemplatehtml for details
76 Request Data in Responses
761 Text Manipulation
It is possible to pull data from the stub request and put it in the response Say you have 200 stubs that are identicalapart from a userid They could be condensed into one stub which uses the userid from the request in the responserequest
ltuseridgtabc123ltuseridgt
matcher
ltuseridgt
response template (using Python)
Hello to request_text[814]
response
Hello to abc123
762 XML Manipulation
Sometimes it is easier to use XPATH to pull data from a request and place it in a response If you need the SessionIdfrom this request
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtJohn1234ltwbsSessionIdgt
Pull the SessionId from the request and use it within the response template
76 Request Data in Responses 19
Stub-O-Matic Documentation Release 0X beta
set namespaces = soapenv httpschemasxmlsoaporgsoapenvelopewbs httpxmlaaacomws200901WBS_Session-20xsd
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtxmltreexpath(soapenvHeaderwbsSessionwbsSessionId
namespaces=namespaces)[0]textltwbsSessionIdgt
Note the xmltree variable is the parsed xml request (as an lxml Root object) made available to the template if therequest is valid xml
Another example - no namespaces in the request Request
ltxml version=10 encoding=UTF-8gtltCompensateCustomersCheck APCWCustom=-1 LocalCurrencyCode=GBP NumberOfAdults=1gt
Response
set currency_code = xmltreexpath(CompensateCustomersCheck)[0]attrib[LocalCurrencyCode] ltresponse LocalCurrencyCode=currency_codegtpay meltresponsegt
77 Stateful Stubs
State is important in simulating back-end systems For example if one was to check-in passenger Bob on a particularflight the back-end would allow this request to succeed only one time and subsequent attempts to check-in the samepassenger on the same flight would be rejected Stubo can emulate this functionality or if desired for repeatable testsStubo can allow Bob to be checked in multiple times on the same flight
Stateful stubbing is enabled automatically If stubs are recorded or loaded in which the same request returns differentresponses Stubo will remember this and internally keep track of which response should come next
If your test is to check-in Bob multiple times simply do not load the stubs representing the check-in rejection andStubo will allow multiple check-ins
78 User Exits
Stub-O-Matic provides hooks into the runtime to execute custom user code Stubo can execute your custom code totransform stub matchers requests andor responses This is a powerful mechanism when your stubs contain dynamicdata or need to change over time This can be useful to perform lsquointelligent stubbingrsquo for example to roll dates Theuser exit API is contained in the stuboextuser_exit module
781 API
782 XML Auto Mangling
If your data is XML a convenience class XMLUserExit is provided to enable auto mangling
20 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
783 API
79 Hooks
710 Modules
Modules are added with the putmodule command and referenced via the ext_module url arg in the putstub call Forexample
putmodulename=staticcmdstestsextxsltmanglerpydeletestubsscenario=mangler_xsltbeginsessionscenario=mangler_xsltampsession=mangler_xslt1ampmode=recordputstubsession=mangler_xslt1ampext_module=mangler firstrequest firstresponseendsessionsession=mangler_xslt1
This example dynamically imports a test mangler from the Stubo server Typically these external python modules thathook into the Stubo runtime would be sourced from source control system using a URL
Note user exit processing is only enabled if the lsquoext_modulersquo variable is set to a pre-loaded module during the putstubcall Each Stub-O-Matic host URL has its own copy of any loaded modules Such modules are available to any testand as such should typically be loaded once per test suite
711 Splitting
Useful to removing random elements from matchers It can happen that the system under test will include seeminglyrandom elements in the request This will cause the next run of the test to not match This problem can be solved byprocessing matchers when loading - during a record A matcher can be split into two with the variable text left outFuture request must match both of the matchers to get the response An example is
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N11111234567806P1ltCommandgtltCryptic_GetScreen_Querygt
Which becomes two matchers
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N1111
and
06P1ltCommandgtltCryptic_GetScreen_Querygt
cutting out the random number lsquo12345678rsquo in the middle
This matcher splitting can be done manually or automatically with a user exit Seestaticcmdstestsextsplitters1firstcommands for and example It uses the rules module at staticcmdstestsextaaapy
Multiple matchers excluding random request elements can be created manually and loaded as such
putstubsession=splitter_record splitter_1textMatcher splitter_2textMatcher splitterresponse
712 Caching Values
If emulating back-end behaviour means writing some code for a particularly tricky behaviour Stubo exposes a simplekey-value cache API to matchers and responses
79 Hooks 21
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
Stub-O-Matic Documentation Release 0X beta
73 Body contains matching
One or more matchers can be defined whereby all matchers must be contained in the request to return a specifiedresponse A typical difficulty with system requests is that they often contain superfluous information such as sessionIDs or time stamps that get in the way of matching a request to the desired response One way Stub-O-Matic can solvethis problem is by the use of lsquomatchersrsquo Matchers are parts of the request that matter to you only what is needed tofind the correct response file
For example a request may include
ltsessionidgt123456ltsessionidgtlttransactionidgt0987654321-9999lttransactionidgt
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Of these 4 lines only the last 2 matter the first 2 lines will differ from one request to the next and be set by the systemmaking the request This request then will be different each time it is made How can one easily match a request likethis to a canned response file without writing code for each requestresponse pair The solution used by Stub-O-Maticis to not store requestresponse file pairs but rather store matchers and the response In the case above the matcherswould be
ltdeparturestartdatetimegt2009-10-24T000000+0000ltdeparturestartdatetimegtltflightnumbergt0455ltflightnumbergt
Stub-O-Matic contains code which will take a request as input search for the response with the best matchers Everymatcher must be contained in the request to be a match When a match is found the corresponding response is servedback
All matchers must be present in the request to make a match Stub-O-Matic will return the response from the firstmatch if there is more than one possible match Note that whitespace and carriage returns are ignored when matching
Alternatives to removing parts of the request you donrsquot want to match on are
1 using a matcher template to substitute ids or dates etc from the request into the matcher OR
2 using a user exit to run custom code A convenience class XMLManglerExit is provided for XML payloads toignore XMl elements andor attributes from the matching process
74 Templated Matcher
This is an example of a template matcher than extracts values from the request into the matcher The xmltree object isan lxml parsed root element passed into the template
ltrequestgtlttransidgtxmltreexpath(requesttransid)[0]textlttransidgtltdtgt2009-10-24T000000+0000ltdtgtltflightnumbergt0455ltflightnumbergtltrequestgt
75 Templated Responses
A hand crafted Stubo response may contain variables which are substituted in For example a response may contain
set flight_nbr=XX1234 The flight number is flight_nbr
18 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
The resulting response served wil be
The flight number is XX1234
Code may be used for example adding random elements to a response
import random set rnd_nbr = randomrandint(1200) Hello World 1000 + rnd_nbr
The resulting response served will be
Hello World - followed by a number between 1000 and 1200
To put todayrsquos date in a response include the following
today_strformat(dmy)
Stubo responses are run through a Tornado template and any logic or commands allowed in these templates can beused See httpwwwtornadoweborgenstabletemplatehtml for details
76 Request Data in Responses
761 Text Manipulation
It is possible to pull data from the stub request and put it in the response Say you have 200 stubs that are identicalapart from a userid They could be condensed into one stub which uses the userid from the request in the responserequest
ltuseridgtabc123ltuseridgt
matcher
ltuseridgt
response template (using Python)
Hello to request_text[814]
response
Hello to abc123
762 XML Manipulation
Sometimes it is easier to use XPATH to pull data from a request and place it in a response If you need the SessionIdfrom this request
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtJohn1234ltwbsSessionIdgt
Pull the SessionId from the request and use it within the response template
76 Request Data in Responses 19
Stub-O-Matic Documentation Release 0X beta
set namespaces = soapenv httpschemasxmlsoaporgsoapenvelopewbs httpxmlaaacomws200901WBS_Session-20xsd
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtxmltreexpath(soapenvHeaderwbsSessionwbsSessionId
namespaces=namespaces)[0]textltwbsSessionIdgt
Note the xmltree variable is the parsed xml request (as an lxml Root object) made available to the template if therequest is valid xml
Another example - no namespaces in the request Request
ltxml version=10 encoding=UTF-8gtltCompensateCustomersCheck APCWCustom=-1 LocalCurrencyCode=GBP NumberOfAdults=1gt
Response
set currency_code = xmltreexpath(CompensateCustomersCheck)[0]attrib[LocalCurrencyCode] ltresponse LocalCurrencyCode=currency_codegtpay meltresponsegt
77 Stateful Stubs
State is important in simulating back-end systems For example if one was to check-in passenger Bob on a particularflight the back-end would allow this request to succeed only one time and subsequent attempts to check-in the samepassenger on the same flight would be rejected Stubo can emulate this functionality or if desired for repeatable testsStubo can allow Bob to be checked in multiple times on the same flight
Stateful stubbing is enabled automatically If stubs are recorded or loaded in which the same request returns differentresponses Stubo will remember this and internally keep track of which response should come next
If your test is to check-in Bob multiple times simply do not load the stubs representing the check-in rejection andStubo will allow multiple check-ins
78 User Exits
Stub-O-Matic provides hooks into the runtime to execute custom user code Stubo can execute your custom code totransform stub matchers requests andor responses This is a powerful mechanism when your stubs contain dynamicdata or need to change over time This can be useful to perform lsquointelligent stubbingrsquo for example to roll dates Theuser exit API is contained in the stuboextuser_exit module
781 API
782 XML Auto Mangling
If your data is XML a convenience class XMLUserExit is provided to enable auto mangling
20 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
783 API
79 Hooks
710 Modules
Modules are added with the putmodule command and referenced via the ext_module url arg in the putstub call Forexample
putmodulename=staticcmdstestsextxsltmanglerpydeletestubsscenario=mangler_xsltbeginsessionscenario=mangler_xsltampsession=mangler_xslt1ampmode=recordputstubsession=mangler_xslt1ampext_module=mangler firstrequest firstresponseendsessionsession=mangler_xslt1
This example dynamically imports a test mangler from the Stubo server Typically these external python modules thathook into the Stubo runtime would be sourced from source control system using a URL
Note user exit processing is only enabled if the lsquoext_modulersquo variable is set to a pre-loaded module during the putstubcall Each Stub-O-Matic host URL has its own copy of any loaded modules Such modules are available to any testand as such should typically be loaded once per test suite
711 Splitting
Useful to removing random elements from matchers It can happen that the system under test will include seeminglyrandom elements in the request This will cause the next run of the test to not match This problem can be solved byprocessing matchers when loading - during a record A matcher can be split into two with the variable text left outFuture request must match both of the matchers to get the response An example is
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N11111234567806P1ltCommandgtltCryptic_GetScreen_Querygt
Which becomes two matchers
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N1111
and
06P1ltCommandgtltCryptic_GetScreen_Querygt
cutting out the random number lsquo12345678rsquo in the middle
This matcher splitting can be done manually or automatically with a user exit Seestaticcmdstestsextsplitters1firstcommands for and example It uses the rules module at staticcmdstestsextaaapy
Multiple matchers excluding random request elements can be created manually and loaded as such
putstubsession=splitter_record splitter_1textMatcher splitter_2textMatcher splitterresponse
712 Caching Values
If emulating back-end behaviour means writing some code for a particularly tricky behaviour Stubo exposes a simplekey-value cache API to matchers and responses
79 Hooks 21
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
Stub-O-Matic Documentation Release 0X beta
The resulting response served wil be
The flight number is XX1234
Code may be used for example adding random elements to a response
import random set rnd_nbr = randomrandint(1200) Hello World 1000 + rnd_nbr
The resulting response served will be
Hello World - followed by a number between 1000 and 1200
To put todayrsquos date in a response include the following
today_strformat(dmy)
Stubo responses are run through a Tornado template and any logic or commands allowed in these templates can beused See httpwwwtornadoweborgenstabletemplatehtml for details
76 Request Data in Responses
761 Text Manipulation
It is possible to pull data from the stub request and put it in the response Say you have 200 stubs that are identicalapart from a userid They could be condensed into one stub which uses the userid from the request in the responserequest
ltuseridgtabc123ltuseridgt
matcher
ltuseridgt
response template (using Python)
Hello to request_text[814]
response
Hello to abc123
762 XML Manipulation
Sometimes it is easier to use XPATH to pull data from a request and place it in a response If you need the SessionIdfrom this request
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtJohn1234ltwbsSessionIdgt
Pull the SessionId from the request and use it within the response template
76 Request Data in Responses 19
Stub-O-Matic Documentation Release 0X beta
set namespaces = soapenv httpschemasxmlsoaporgsoapenvelopewbs httpxmlaaacomws200901WBS_Session-20xsd
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtxmltreexpath(soapenvHeaderwbsSessionwbsSessionId
namespaces=namespaces)[0]textltwbsSessionIdgt
Note the xmltree variable is the parsed xml request (as an lxml Root object) made available to the template if therequest is valid xml
Another example - no namespaces in the request Request
ltxml version=10 encoding=UTF-8gtltCompensateCustomersCheck APCWCustom=-1 LocalCurrencyCode=GBP NumberOfAdults=1gt
Response
set currency_code = xmltreexpath(CompensateCustomersCheck)[0]attrib[LocalCurrencyCode] ltresponse LocalCurrencyCode=currency_codegtpay meltresponsegt
77 Stateful Stubs
State is important in simulating back-end systems For example if one was to check-in passenger Bob on a particularflight the back-end would allow this request to succeed only one time and subsequent attempts to check-in the samepassenger on the same flight would be rejected Stubo can emulate this functionality or if desired for repeatable testsStubo can allow Bob to be checked in multiple times on the same flight
Stateful stubbing is enabled automatically If stubs are recorded or loaded in which the same request returns differentresponses Stubo will remember this and internally keep track of which response should come next
If your test is to check-in Bob multiple times simply do not load the stubs representing the check-in rejection andStubo will allow multiple check-ins
78 User Exits
Stub-O-Matic provides hooks into the runtime to execute custom user code Stubo can execute your custom code totransform stub matchers requests andor responses This is a powerful mechanism when your stubs contain dynamicdata or need to change over time This can be useful to perform lsquointelligent stubbingrsquo for example to roll dates Theuser exit API is contained in the stuboextuser_exit module
781 API
782 XML Auto Mangling
If your data is XML a convenience class XMLUserExit is provided to enable auto mangling
20 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
783 API
79 Hooks
710 Modules
Modules are added with the putmodule command and referenced via the ext_module url arg in the putstub call Forexample
putmodulename=staticcmdstestsextxsltmanglerpydeletestubsscenario=mangler_xsltbeginsessionscenario=mangler_xsltampsession=mangler_xslt1ampmode=recordputstubsession=mangler_xslt1ampext_module=mangler firstrequest firstresponseendsessionsession=mangler_xslt1
This example dynamically imports a test mangler from the Stubo server Typically these external python modules thathook into the Stubo runtime would be sourced from source control system using a URL
Note user exit processing is only enabled if the lsquoext_modulersquo variable is set to a pre-loaded module during the putstubcall Each Stub-O-Matic host URL has its own copy of any loaded modules Such modules are available to any testand as such should typically be loaded once per test suite
711 Splitting
Useful to removing random elements from matchers It can happen that the system under test will include seeminglyrandom elements in the request This will cause the next run of the test to not match This problem can be solved byprocessing matchers when loading - during a record A matcher can be split into two with the variable text left outFuture request must match both of the matchers to get the response An example is
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N11111234567806P1ltCommandgtltCryptic_GetScreen_Querygt
Which becomes two matchers
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N1111
and
06P1ltCommandgtltCryptic_GetScreen_Querygt
cutting out the random number lsquo12345678rsquo in the middle
This matcher splitting can be done manually or automatically with a user exit Seestaticcmdstestsextsplitters1firstcommands for and example It uses the rules module at staticcmdstestsextaaapy
Multiple matchers excluding random request elements can be created manually and loaded as such
putstubsession=splitter_record splitter_1textMatcher splitter_2textMatcher splitterresponse
712 Caching Values
If emulating back-end behaviour means writing some code for a particularly tricky behaviour Stubo exposes a simplekey-value cache API to matchers and responses
79 Hooks 21
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
Stub-O-Matic Documentation Release 0X beta
set namespaces = soapenv httpschemasxmlsoaporgsoapenvelopewbs httpxmlaaacomws200901WBS_Session-20xsd
ltxml version=10 encoding=UTF-8gtltsoapenvEnvelope xmlnssoapenv=httpschemasxmlsoaporgsoapenvelope
xmlnswbs=httpxmlaaacomws200901WBS_Session-20xsdgtltsoapenvHeadergt
ltwbsSessiongtltwbsSessionIdgtxmltreexpath(soapenvHeaderwbsSessionwbsSessionId
namespaces=namespaces)[0]textltwbsSessionIdgt
Note the xmltree variable is the parsed xml request (as an lxml Root object) made available to the template if therequest is valid xml
Another example - no namespaces in the request Request
ltxml version=10 encoding=UTF-8gtltCompensateCustomersCheck APCWCustom=-1 LocalCurrencyCode=GBP NumberOfAdults=1gt
Response
set currency_code = xmltreexpath(CompensateCustomersCheck)[0]attrib[LocalCurrencyCode] ltresponse LocalCurrencyCode=currency_codegtpay meltresponsegt
77 Stateful Stubs
State is important in simulating back-end systems For example if one was to check-in passenger Bob on a particularflight the back-end would allow this request to succeed only one time and subsequent attempts to check-in the samepassenger on the same flight would be rejected Stubo can emulate this functionality or if desired for repeatable testsStubo can allow Bob to be checked in multiple times on the same flight
Stateful stubbing is enabled automatically If stubs are recorded or loaded in which the same request returns differentresponses Stubo will remember this and internally keep track of which response should come next
If your test is to check-in Bob multiple times simply do not load the stubs representing the check-in rejection andStubo will allow multiple check-ins
78 User Exits
Stub-O-Matic provides hooks into the runtime to execute custom user code Stubo can execute your custom code totransform stub matchers requests andor responses This is a powerful mechanism when your stubs contain dynamicdata or need to change over time This can be useful to perform lsquointelligent stubbingrsquo for example to roll dates Theuser exit API is contained in the stuboextuser_exit module
781 API
782 XML Auto Mangling
If your data is XML a convenience class XMLUserExit is provided to enable auto mangling
20 Chapter 7 Stubbing
Stub-O-Matic Documentation Release 0X beta
783 API
79 Hooks
710 Modules
Modules are added with the putmodule command and referenced via the ext_module url arg in the putstub call Forexample
putmodulename=staticcmdstestsextxsltmanglerpydeletestubsscenario=mangler_xsltbeginsessionscenario=mangler_xsltampsession=mangler_xslt1ampmode=recordputstubsession=mangler_xslt1ampext_module=mangler firstrequest firstresponseendsessionsession=mangler_xslt1
This example dynamically imports a test mangler from the Stubo server Typically these external python modules thathook into the Stubo runtime would be sourced from source control system using a URL
Note user exit processing is only enabled if the lsquoext_modulersquo variable is set to a pre-loaded module during the putstubcall Each Stub-O-Matic host URL has its own copy of any loaded modules Such modules are available to any testand as such should typically be loaded once per test suite
711 Splitting
Useful to removing random elements from matchers It can happen that the system under test will include seeminglyrandom elements in the request This will cause the next run of the test to not match This problem can be solved byprocessing matchers when loading - during a record A matcher can be split into two with the variable text left outFuture request must match both of the matchers to get the response An example is
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N11111234567806P1ltCommandgtltCryptic_GetScreen_Querygt
Which becomes two matchers
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N1111
and
06P1ltCommandgtltCryptic_GetScreen_Querygt
cutting out the random number lsquo12345678rsquo in the middle
This matcher splitting can be done manually or automatically with a user exit Seestaticcmdstestsextsplitters1firstcommands for and example It uses the rules module at staticcmdstestsextaaapy
Multiple matchers excluding random request elements can be created manually and loaded as such
putstubsession=splitter_record splitter_1textMatcher splitter_2textMatcher splitterresponse
712 Caching Values
If emulating back-end behaviour means writing some code for a particularly tricky behaviour Stubo exposes a simplekey-value cache API to matchers and responses
79 Hooks 21
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
Stub-O-Matic Documentation Release 0X beta
783 API
79 Hooks
710 Modules
Modules are added with the putmodule command and referenced via the ext_module url arg in the putstub call Forexample
putmodulename=staticcmdstestsextxsltmanglerpydeletestubsscenario=mangler_xsltbeginsessionscenario=mangler_xsltampsession=mangler_xslt1ampmode=recordputstubsession=mangler_xslt1ampext_module=mangler firstrequest firstresponseendsessionsession=mangler_xslt1
This example dynamically imports a test mangler from the Stubo server Typically these external python modules thathook into the Stubo runtime would be sourced from source control system using a URL
Note user exit processing is only enabled if the lsquoext_modulersquo variable is set to a pre-loaded module during the putstubcall Each Stub-O-Matic host URL has its own copy of any loaded modules Such modules are available to any testand as such should typically be loaded once per test suite
711 Splitting
Useful to removing random elements from matchers It can happen that the system under test will include seeminglyrandom elements in the request This will cause the next run of the test to not match This problem can be solved byprocessing matchers when loading - during a record A matcher can be split into two with the variable text left outFuture request must match both of the matchers to get the response An example is
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N11111234567806P1ltCommandgtltCryptic_GetScreen_Querygt
Which becomes two matchers
ltCryptic_GetScreen_QuerygtltCommandgtFXXNFCDXO6767820099000077061214N1111
and
06P1ltCommandgtltCryptic_GetScreen_Querygt
cutting out the random number lsquo12345678rsquo in the middle
This matcher splitting can be done manually or automatically with a user exit Seestaticcmdstestsextsplitters1firstcommands for and example It uses the rules module at staticcmdstestsextaaapy
Multiple matchers excluding random request elements can be created manually and loaded as such
putstubsession=splitter_record splitter_1textMatcher splitter_2textMatcher splitterresponse
712 Caching Values
If emulating back-end behaviour means writing some code for a particularly tricky behaviour Stubo exposes a simplekey-value cache API to matchers and responses
79 Hooks 21
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
Stub-O-Matic Documentation Release 0X beta
For example if a response needs to increment a count each time it is used you can define a module like this
import loggingfrom stuboextuser_exit import GetResponse ExitResponse
log = logginggetLogger(__name__)
class IncResponse(GetResponse)Increments a value in the response using the provided cachedef __init__(self request context)
GetResponse__init__(self request context)
def inc(self)cache = selfcontext[cache]val = cacheget(foo)logdebug(cache val = 0format(val))if not val
val = 0val += 1cacheset(foo val)return val
def doResponse(self)stub = selfcontext[stub]stubset_response_body(ltresponsegt0ltresponsegtformat(selfinc()))return ExitResponse(selfrequest stub)
def exits(request context)if context[function] == getresponse
return IncResponse(request context)
The lines above to note are
val = cacheget(foo)
cacheset(foo val)
You can set any key and value which is of use As with other user extensions the example above can be loaded in acommand with
putmodulename=staticcmdstestsextcacheexamplepyputstubsession=cache_1ampext_module=exampleamptracking_level=full1request1response
22 Chapter 7 Stubbing
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
CHAPTER 8
Sharing Stubs
Stub-O-Matic allows stubs to be organised and shared in ways to suit a projectrsquos test data needs Stubo is aboutemulating back-end system behaviour and as such stubs can be organised around individual services provided by agiven back-end service These stubs can be assembled and re-used as required by individual tests
Another option is for each test to own its own stubs This makes each test and its data independent from all others butcan also lead to an explosion of stubs and potential management issues if they need to be upgraded to changes in theback-end system
As an example consider an end to end use case for a Game Leader application as follows Test 4 - Change leader
GIVEN current (10 July) leader is Bob
WHEN current leader is requested
THEN leader is Bob
WHEN Sue enters a score of 1000 for 10 July
WHEN current leader is requested
THEN leader is now Sue
This test uses both the increment_score service and the view_leader service Many stubs from the other tests can bereused and will be by creating a new command file
Stubo command files can use stubs from folders relative to the command file location or by URL Refer to the lsquoCom-mandsrsquo section of this documentation for details
change_setupcommands
load and test stubs for change leader testdeletestubsscenario=change_leaderbeginsessionscenario=change_leaderampsession=change_leaderampmode=record
add points to bob on the 10th Julyputstubsession=change_leaderincrementincr_bob_10th_6textMatcherincrementincr_bob_10th_6response
find leader on 10th ( it is bob)putstubsession=change_leaderleaderlead_1textMatcherleaderlead_1response
add 1000 points to Sueputstubsession=change_leaderincr_sue_10th_1000textMatcherincr_sue_10th_1000response
find leader on 10th ( it is now Sue)putstubsession=change_leaderleaderlead_1textMatcherleader_is_sueresponse
23
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
Stub-O-Matic Documentation Release 0X beta
endsessionsession=change_leader
beginsessionscenario=change_leaderampsession=change_leaderampmode=playback
change_responsescommands
increment Bobs score by 6 points first entry for day so should return 6getresponsesession=change_leaderincrementincr_bob_10th_6request
find the leadergetresponsesession=change_leaderleaderlead_1request
increment Sues score by 1000 pointsgetresponsesession=change_leaderincr_sue_10th_1000request
find the leadergetresponsesession=change_leaderleaderlead_1request
change_teardowncommand
endsessionsession=change_leaderdeletestubsscenario=change_leader
incr_sue_10th_1000request
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
incr_sue_10th_1000textMatcher
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltincrementbygt1000ltincrementbygt
incr_sue_10th_1000response
ltincrementScoregtltdategt2013-07-10ltdategtltcontestantgtsueltcontestantgtltnew_scoregt1000ltnew_scoregtltauthentication_codegt23455672213456ltauthentication_codegtltgameIdgt1234ltgameIdgt
ltincrementScoregt
leader_is_sueresponse
ltleadergtltboardIdgt1234ltboardIdgt
ltcontestantgtsueltcontestant if stubbedTodayday == 1 ltprevMonthWinnergtsallyltprevMonthWinnergt end
ltleadergt
24 Chapter 8 Sharing Stubs
Stub-O-Matic Documentation Release 0X beta
These stubs and command files show how Stubo can mingle stubs provided at a service function level with stubsspecific to a given test
25
Stub-O-Matic Documentation Release 0X beta
26 Chapter 8 Sharing Stubs
CHAPTER 9
Communicating with Stub-O-Matic
Although Stub-O-Matic has a GUI it is best used for automated repeatable interactions with systems and test toolsLoading calling and deleting of stubs should all be automated Ideally your tests should be able to run overnightwithout human interaction
91 GetResponse URLs
Generally Stubo uses URL arguments such as POST
httpserverstuboapigetresponsesession=123
Additionally a getresponse call can be POST
httpserverstuboapigetresponseYOURRANDOMURL
with the Stubo session in an HTTP Header xx_stb_session=123 (where xx can be any two characters) In both casesthe request text is in the HTTP body
This allows request to be sent directly to Stubo from some systems without going through a Stubo integrator
92 Virtual Stub-O-Matic
One Stubo server or cluster can support many tests test environments and teams To help teams and users view onlythe stubs and sessions they are interested in a single physical Stubo server can be addressed by different URLs
For example httpstubo_1 and httpstubo_2 can point to the same physical machine Test team 1 could load asession lsquotest123rsquo on httpstubo_1 and test team 2 could load an identically named session on httpstubo_2 Thesetwo session would be treated completely independently by Stubo as if they were on different hardware
This allows for more efficient use of hardware and less software management overhead
Stub-O-Matic REST API v2
Stubo REST API v2 returns JSON The response always returns the version and payload Responses are similar to v1API responses The payload is either contained under lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errorscontain a descriptive message under lsquomessagersquo and the http error code under lsquocodersquo Successful responses depend onthe call made and are described below
Encoding ldquoapplicationjsonrdquo
Create scenario object
27
Stub-O-Matic Documentation Release 0X beta
Creates scenario object Client must specify scenario name in the request body
bull __URL__ stuboapiv2scenarios
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario created
ndash __422__ - scenario with that name already exists
ndash __400__ - something is missing (eg name)
bull __Example request body__
lsquolsquolsquojavascript
ldquoscenariordquo ldquoscenario_namerdquo
28 Chapter 9 Communicating with Stub-O-Matic
CHAPTER 10
Get scenario list
Returns a list of scenarios with their name and scenarioRef attributes This attribute can the be used to get scenariodetails (sessions stubs size etc) Your application can look for these keys and use them to directly access resourceinstead of generating URL on client side
bull __URL__ stuboapiv2scenarios
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [ ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquonamerdquo ldquolocal-hostscenario_1rdquo ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquo
ldquonamerdquo ldquolocalhostscenario_10rdquo ]
29
Stub-O-Matic Documentation Release 0X beta
30 Chapter 10
CHAPTER 11
Get scenario list with details
Returns a list of scenarios with details
bull __URL__ stuboapiv2scenariosdetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example output__
lsquolsquolsquojavascript
ldquodatardquo [
ldquospace_used_kbrdquo 0 ldquonamerdquo ldquolocalhostscenario_1rdquo ldquosessionsrdquo [] ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_1rdquo ldquorecordedrdquo ldquo-rdquo ldquostub_countrdquo 0
ldquospace_used_kbrdquo 544 ldquonamerdquo ldquolocalhostscenario_10rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_10rdquoldquolast_usedrdquo ldquo2015-07-20 130905rdquo
]
ldquoscenarioRefrdquo ldquostuboapiv2scenariosobjectslocalhostscenario_10rdquoldquorecordedrdquo ldquo2015-07-20rdquo ldquostub_countrdquo 20
]
31
Stub-O-Matic Documentation Release 0X beta
32 Chapter 11
CHAPTER 12
Get scenario details
Returns specified scenario details Scenario name can be provided with a host for examplestubo_c1yourcorporateaddresscomscenario_name_x
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ GET
bull __Response codes__
ndash __200__ - specified scenario details
ndash __404__ - specified scenario not found
bull __Example output__
lsquolsquolsquojavascript
ldquospace_used_kbrdquo 697 ldquonamerdquo ldquolocalhostscenario_13rdquo ldquosessionsrdquo [
ldquostatusrdquo ldquoplaybackrdquo ldquoloadedrdquo ldquo2015-07-20rdquo ldquonamerdquo ldquoplayback_13rdquo
ldquolast_usedrdquo ldquo2015-07-20 130905rdquo
] ldquostubsrdquo 26 ldquorecordedrdquo ldquo2015-07-20rdquo ldquoscenarioRefrdquoldquostuboapiv2scenariosobjectslocalhostscenario_13rdquo
lsquolsquolsquo
Delete scenario
Deletes scenario object and removed itrsquos stubs from cache
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - scenario deleted
ndash __412__ - precondition failed - specified scenario does not exist
Begin session and set mode
Begins session for specified scenario Client has to specify session name and mode in request body Session mode canbe either lsquorecordrsquo and lsquoplaybackrsquo
33
Stub-O-Matic Documentation Release 0X beta
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - begins session + __400__ - something went wrong (eg session alreadyexists)
bull __Example request body__
lsquolsquolsquojavascript
ldquobeginrdquo null ldquosessionrdquo ldquosession_namerdquo ldquomoderdquo ldquorecordrdquo
34 Chapter 12
CHAPTER 13
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquoerrorrdquo
ldquomessagerdquo ldquoScenario (localhostscenario_10) has existing stubs delete them before record-ing or create another scenariordquo ldquocoderdquo 400
35
Stub-O-Matic Documentation Release 0X beta
36 Chapter 13
CHAPTER 14
End session
Ends specified session Client has to specify session name in request body
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__ + __200__ - session ended
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo null ldquosessionrdquo ldquoplayback_28rdquo
37
Stub-O-Matic Documentation Release 0X beta
38 Chapter 14
CHAPTER 15
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
End all sessions for specific scenario
Ends all sessions for specified scenario
bull __URL__ stuboapiv2scenariosobjectsltscenario_namegtaction
bull __Method__ POST
bull __Response codes__
ndash __200__ - scenario list with details returned
bull __Example request body__
lsquolsquolsquojavascript
ldquoendrdquo ldquosessionsrdquo
39
Stub-O-Matic Documentation Release 0X beta
40 Chapter 15
CHAPTER 16
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquoplayback_20rdquo ldquomessagerdquo ldquoSession endedrdquo
lsquolsquolsquo
Get delay policy list
Gets all defined delay policies
bull __URL__ stuboapiv2delay-policydetail
bull __Method__ GET
bull __Response codes__
ndash __200__ - list with delay policies returned
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
ldquodelay_typerdquo ldquofixed_2rdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquo
ldquonamerdquo ldquoslowerrdquo ldquomillisecondsrdquo ldquo5000rdquo
lsquolsquolsquo
Get specific delay policy details
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ GET
41
Stub-O-Matic Documentation Release 0X beta
bull __Response codes__
ndash __200__ - delay policy returned
ndash __404__ - delay policy not found
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo [
ldquodelay_typerdquo ldquofixedrdquo ldquodelayPolicyRefrdquo ldquostuboapiv2delay-policyobjectsslowrdquoldquonamerdquo ldquoslowrdquo ldquomillisecondsrdquo ldquo1000rdquo
]
42 Chapter 16
CHAPTER 17
Add delay policy
Creates a delay policy Returns 201 status code if successful or 409 if request body options are wrong (type fixedprovided with mean and stddev options)
bull __URL__ stuboapiv2delay-policy
bull __Method__ PUT
bull __Response codes__
ndash __201__ - scenario list with details returned
ndash __400__ - bad request
ndash __409__ - wrong combination of options was used
bull __Example request body__
lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquofixedrdquo ldquomillisecondsrdquo 50
43
Stub-O-Matic Documentation Release 0X beta
44 Chapter 17
CHAPTER 18
or lsquolsquolsquojavascript
ldquonamerdquo ldquodelay_namerdquo ldquodelay_typerdquo ldquonormalvariaterdquo ldquomeanrdquo ldquomean_valuerdquo ldquostddevrdquo ldquovalrdquo
45
Stub-O-Matic Documentation Release 0X beta
46 Chapter 18
CHAPTER 19
or
lsquolsquolsquojavascript
ldquonamerdquo ldquoweighted_delay_namerdquo ldquodelay_typerdquo ldquoweightedrdquo
ldquodelaysrdquo ldquofixed300005normalvariate5000100015normalvariate100050070rdquo
lsquolsquolsquo
bull __Example output__
lsquolsquolsquojavascript
ldquostatus_coderdquo 201 ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquostatusrdquo ldquonewrdquo ldquomessagerdquo ldquoPut Delay Policy Finishedrdquo ldquodelay_typerdquo ldquofixedrdquo ldquodelayPoli-cyRefrdquo ldquostuboapiv2delay-policyobjectsmy_delayrdquo ldquonamerdquo ldquomy_delayrdquo
lsquolsquolsquo
Delete delay policy
bull __URL__ stuboapiv2delay-policyobjectsltnamegt
bull __Method__ DELETE
bull __Response codes__
ndash __200__ - delay policy deleted
bull __Example output__
lsquolsquolsquojavascript
ldquoversionrdquo ldquo066rdquo ldquodatardquo
ldquomessagerdquo ldquoDeleted 1 delay policies from [ursquomy_delayrsquo]rdquo
47
Stub-O-Matic Documentation Release 0X beta
48 Chapter 19
CHAPTER 20
49
Stub-O-Matic Documentation Release 0X beta
50 Chapter 20
CHAPTER 21
Stub-O-Matic API v1
The Stubo API v1 returns JSON The response always returns the version and payload The payload is either containedunder lsquodatarsquo if the response is successful or lsquoerrorrsquo for errors Errors contain a descriptive message under lsquomessagersquoand the http error code under lsquocodersquo Successful responses depend on the call made and are described below
eg an error response
version 123error
message Session already exists - localhostfirstfirst_1 in playback modecode 400
eg a successful response
version 123data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
211 execcmds
execcmds (GET POST)query args
cmdfile URL or a file under static on the stubo server+ any user args will be made avaliable to the cmd file templatesession_id session name to substitute within the cmdfile template best to prefix this with scenario name if provided (optional)
response shows the list of commands (url return_code) executed see the Tracker page for responses
Typically command files are used to load stubs into the Stubo db but you can run any supported commands from a file
stuboapiexeccmdscmdfile=staticcmdsdemofirstcommands
51
Stub-O-Matic Documentation Release 0X beta
version 123data
executed_commands [[
deletestubsscenario=first200
][
beginsessionscenario=firstampsession=first_1ampmode=record200
][
putstubsession=first_1firsttextMatcherfirstresponse200
][
endsessionsession=first_1200
][
beginsessionscenario=firstampsession=first_1ampmode=playback200
][
getresponsesession=first_1firstrequest200
][
endsessionsession=first_1200
]]number_of_requests 7number_of_errors 0
stuboapiexeccmdscmdfile=httpsyour-source-repomycommands
Command files can be included in an archive file along with the matcher and response files You can import your stubsby running the archive file
stuboapiexeccmdscmdfile=httpsyour-source-repomyzip
Supported archive formats are zip targz amp jar files
212 getversion
getversion (GET POST)
This call does not touch the db or cache so is useful as a quick ping on the server
stuboapigetversion
52 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 123
213 getstatus
getstatus (GET POST)query args
scenario=namesession=name (session takes precedence)check_database=true|false (default true)local_cache=true|false (default true)
stuboapigetstatusscenario=first
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
database_server
status oksessions [
[first_1dormant
]]
stuboapigetstatussession=first_1
version 123data
cache_server status oklocal true
info
cluster my-clustergraphite_host httpmy-graphitecom
session
status dormantsystem_date 2014-10-02scenario localhostfirstlast_used 2014-10-02 160039scenario_id 542d76a7ac5f73060fc9c2b4session first_1
213 getstatus 53
Stub-O-Matic Documentation Release 0X beta
database_server
status ok
214 beginsession
beginsession (GET POST)query args
scenario = scenario namesession = session namemode = playback|record
stuboapibeginsessionscenario=firstampsession=first_1ampmode=playback
version 599data
status playbackmessage Playback mode initiatedsession first_1scenario localhostfirst
Note on duplicate scenarios and sessions
A scenario name prefixed with the stubo host name must be unique One cannot record a new scenario with a duplicate host + scenario name
Sessions are instances of scenarios stubs and must be unique within a host
Sessions can not be deleted if in playback or record mode
Scenarios can not be deleted if any session based on it is in playback or record mode
215 endsession
endsession (GET POST)query args
session session name
stuboapiendsessionsession=first_1
version 123data
message Session ended
Ending a session which does not exist is OK and will complete successfully
54 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
216 endsessions
endsessions (GET POST)query args
scenario scenario name
stuboapiendsessionsscenario=first
version 613data
first_1 message Session ended
first_2
message Session ended
217 putscenarios
Scenario names can be changed by providing current scenario name and new name This operation includes renamingall the stubs that belong to this scenario as well as changing scenario name value in saved sessions Sessions will betransfered to new scenario During rename procedure - all sessions will be set to dormant mode Returns status code412 if no name or no query is provided Returns status code 400 if scenario name has illegal characters Scenarioname check regex rrsquo[w-]$rsquo - letters numbers dashes underscores
putscenarios(Pltscenario_namegt[^]+) (GET)query args
new_name new scenario name
stuboapiputscenariosfirstnew_name=new_first_scenario_name
Scenarios changed 1
Remapped sessions [
name myscenario_session2
]New name localhostnew_first_scenario_nameOld name localhostfirstStubs changed 5Pre stubs changed 0
218 putstub
putstub (POST)query args
session = session name
216 endsessions 55
Stub-O-Matic Documentation Release 0X beta
ext_module = external module name without py extenstion (optional)delay_policy = delay policy name (optional)stateful = treat duplicate stubs as stateful otherwise ignore duplicates if stateful=false (default true optional)tracking_level full or normal (optional overrides host or global setting)+ any user args will be made avaliable to the matcher amp response templates and any user exit code
egstuboapiputstubsession=my_session
given request=ltstatusgtIS_OKltstatusgt amp response=ltresponsegtYESltresponsegtJSON POST data
request method POSTbodyPatterns [
contains [ltstatusgtIS_OKltstatusgt] ]
response status 200body ltresponsegtYESltresponsegt
returns
data message put 54378c0dac5f7302b5cb8e56 stub
version 123
Treatment of duplicate stubs
If both the request and the response already exist for the scenario in record mode then the stub will not be created
If the request exists but with a different response the second response will be recorded and the stub becomes a stateful stub
Duplicate stubs can exist in different scenarios
Notes
see Stub Definition for stub definitions see Templates Date Rolling for an example of using user arguments to performdate rolling
219 getstublist
getstublist (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)
stuboapigetstublistscenario=first
version 123data
stubs [
recorded 2014-10-10
56 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
args session first_1
request
bodyPatterns [
contains [get my stubn
]
]method POST
response
status 200body Hello 1+1 Worldn
]scenario first
2110 putdelay_policy
putdelay_policy (GET POST)query args
name delay namedelay_type fixed normalvariate or weightedmilliseconds used with fixed delay_type onlymean used with normalvariate delay_type onlystddev used with normalvariate delay_type onlyvalues used with weighted delay_type only values is a delimited string of delaysFor each delay the last value represents the percentage this delay will occur
stuboapiputdelay_policyname=slowampdelay_type=fixedampmilliseconds=1000
version 123data
status newmessage Put Delay Policy Finisheddelay_type fixedname slow
ie to set a weighted percentage of delays with 5 fixed at 30s 15 having a delay of 5s +- 1s and 70 having a delay of 1s +- 05sstuboapiputdelay_policyname=pcent_random_samplesampdelay_type=weightedampdelays=fixed300005normalvariate5000100015normalvariate100050070
version 123data
status newmessage Put Delay Policy Finisheddelay_type weightedname pcent_random_samples
2110 putdelay_policy 57
Stub-O-Matic Documentation Release 0X beta
2111 getdelay_policy
getdelay_policy (GET POST)query args
name delay name (optional lists all if not provided)
stuboapigetdelay_policyname=slow
version 123data
slow delay_type fixedname slowmilliseconds 1000
2112 deletedelay_policy
deletedelay_policy (GET POST)query args
name delay name (optional deletes all if not provided)
stuboapideletedelay_policyname=slow
version 123data
message Deleted 1 delay policies from [uslow]
2113 getresponse
getresponse (POST)query args
session session nametracking_level full or normal (optional overrides host or global setting)
POST data request payloadHTTP headers
Stubo-Request-Session=123 Optional can be used in place of session on the URLreturns stub response payload in HTTP body if okon error returns stubo json error response
stuboapigetresponsesession=first_1POST data get my stubreturns Hello 2 World
58 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2114 deletestubs
Stubs should be mastered in a code repository such as SVN Deletestubs will remove stubs from the Stubo databaseThis should be run at the end of each test run
deletestubs (GET POST)query args
scenario scenario namehost host uri to use (defaults to host used in request uri optional)force false or true (optional defaults to false)
stuboapideletestubsscenario=first
version 123data
scenarios [localhostfirst
]message stubs deleted
All sessions must be in a dormant state to delete the stubs unless force=true is used
Deleting a scenario that does not exist is OK and will complete successfully
2115 getexport
Export a recorded scenario To support repeatable testing a recording should be exported with getexport and theresulting archive file saved to your source code repository (GIT etc) The exported archive contains all scenario stubsand a command script to reload them The getexport call also supports exporting lsquorunnablersquo scenarios A lsquorunnablersquoscenario will add a playback of a previous session to the command script This can be useful to compare different testruns with each other
getexport (GET POST)query args
scenario scenario namesession_id session id to use within the export (optional defaults to epoch time)export_dir export dir name (optional defaults to scenario key)runnable create a runnable scenario of a previous playback (optional)playback_session playback session to use (required with runnable)session_id session name to substitute within the cmdfile template (optional)
returns links to exported archive files (zip targz jar)
stuboapigetexportscenario=first
version 123data
scenario firstexport_dir_name Usersrowandeveclipseworkspacestubostaticexportslocalhost_firstlinks [
[first_1412947560_0response0httpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0response0v=1d63737c9cdb7b1433d76b52661c9db9
2114 deletestubs 59
Stub-O-Matic Documentation Release 0X beta
][
first_1412947560_0_0textMatcherhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirst_1412947560_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
firstcommandshttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstcommandsv=d56a304dddafe558ccfe9340ebdb41e8
][
firstziphttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstzipv=34c1c698d09e7e3f1a3a10a2834bbbd6
][
firsttargzhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirsttargzv=8e5ac69d3041941aa4cc5dfdee41326b
][
firstjarhttpRowan-MacBook-Pro-5local8001staticexportslocalhost_firstfirstjarv=34c1c698d09e7e3f1a3a10a2834bbbd6
]]
amp runnable export
stuboapigetexportscenario=firstamprunnable=trueampplayback_session=first_1
version 123data
runnable last_used
start_time 2015-03-24 165703248000+0000remote_ip 1
playback_session first_1number_of_playback_requests 1
scenario firstlinks [
[first_1427285580_0response0httpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0response0v=1d63737c9cdb7b1433d76b52661c9db9
][
first_1427285580_0_0textMatcherhttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0_0textMatcherv=088c16fa5004e2467126cfeaf8da3cd3
][
first_1427285580_0requesthttpvuze-on-pc2home8001staticexportslocalhost_firstfirst_1427285580_0requestv=925721a672115ec9bfc24f55a6979a63
][
firstcommandshttpvuze-on-pc2home8001staticexportslocalhost_firstfirstcommandsv=98ad4927b82478744dfa004f48f88aff
]
60 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
[firstziphttpvuze-on-pc2home8001staticexportslocalhost_firstfirstzipv=66a370b25ca2065abc4deb347ee77ce6
][
firsttargzhttpvuze-on-pc2home8001staticexportslocalhost_firstfirsttargzv=da76a1ce23a9cfe2dc1895955021f3c4
][
firstjarhttpvuze-on-pc2home8001staticexportslocalhost_firstfirstjarv=66a370b25ca2065abc4deb347ee77ce6
]]export_dir_path Usersrowandeveclipseworkspaceopencredostubolateststubo-appstubostaticexportslocalhost_first
2116 getstubcount
getstubcount (GET POST)query args
scenario scenario name (optional)
Returns the number of stubs for a given scenario or all scenarios on host ifthe scenario is not provided
stuboapigetstubcountscenario=first
version 123data
count 1scenario first
2117 putmodule
User exits can be applied to perform custom manipulation of Stubo matchers and responses The user exits are pythoncode defined with the UserExit API The code is input into stubo with the following API call
putmodule (GET POST)query args
name full path to module can be a uri
stuboapiputmodulename=staticcmdstestsextxsltmanglerpy
version 123data
message added modules [localhost_mangler_v1]
2116 getstubcount 61
Stub-O-Matic Documentation Release 0X beta
Notes
If the module code has not changed an error is returned indicating that the source has not changed otherwise a newversion of the module is added to stubo dynamically
2118 getmodulelist
getmodulelist (GET POST)returns list of loaded modules
stuboapigetmodulelist
version 123data
info mangler
loaded_sys_versions [localhost_mangler_v1
]latest_code_version 1
message list modules
2119 deletemodule
Delete named module
deletemodule (GET POST)query args
name name of module without py ext
version 123data
deleted [localhostmangler
]message delete modules [umangler]
2120 deletemodules
Delete all modules from this host URL
deletemodules (GET POST)
62 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
version 613data
deleted [localhoststrip_nslocalhostignore_dates
]message delete modules [strip_ns ignore_dates]
2121 Set Tracking Level
All API calls to Stubo will result in a tracking record being created Default level tracking includes
bull start time
bull duration
bull any user configured delay
bull stubo function
bull return code and data
bull session and scenario names
bull response size
bull server (Stubo server that handled the request)
bull host (DNS of stubo used on the request)
bull remote_ip (IP address of the client)
In addition getresponse calls can optionally force other items to be tracked including
bull matchers used
bull matcher text before during and after any mangling
bull response text before during and after any mangling
To enabledisable logging
putsetting (GET POST)query args
tracking_level=full or normal
stuboapiputsettingsetting=tracking_levelampvalue=full
version 123data
new falsehost localhostall falsetracking_level full
Click on a getresponse item in the Tracker page to see the full tracking data
2121 Set Tracking Level 63
Stub-O-Matic Documentation Release 0X beta
2122 Blacklist a host URL
To stop a virtual stubo host being used perform the following
stuboapiputsettinghost=roguehostampsetting=blacklistedampvalue=on
version 123data
blacklisted onnew truehost roguehostall 0
stuboapigetsettinghost=roguehostampsetting=blacklisted
version 123data
blacklisted onall 0host roguehost
Users will not be able to start a session with this host after it has been blacklisted
roguehoststuboapibeginsession
version 123error
message Sorry the host URL roguehost has been blacklisted Please contact Stub-O-Matic supportcode 400
2123 Create Bookmark
This is usually done via the GUI
Method POST putbookmarkname=abcampsession=bobampsession=maryampresponse=12345URL Variables name=bookmark_name session=session_1ampsession=session_2 response=1234 (response key)Request Body -empty-Returns
64 Chapter 21 Stub-O-Matic API v1
Stub-O-Matic Documentation Release 0X beta
2124 Jump to Bookmark
Method GET jumpbookmarkURL Variables name=bookmark_name session=session_1ampsession=session_2Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquoresultsrdquo [[rdquoe121bef2c162a2ee4ae63rdquo ldquo2rdquo 0]]
2125 Delete Bookmark
Method GET deletebookmarkURL Variables name=bookmark_name scenario=abcRequest Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquobookmarkrdquo ldquobob_leadsrdquo ldquodeletedrdquo 02rdquo 0]]
2126 List Bookmarks
Method GET getbookmarkURLVariables
name=bookmark_name (optionl lists all if absent)
Request Body -empty-Returns ldquoversionrdquo ldquo550rdquo ldquodatardquo ldquotrngrdquo ldquobob_leadsrdquo ldquoe1213rdquo ldquo1rdquo ldquoted_leadsrdquo ldquoe1213brdquo
ldquo2rdquo
2127 getstats
Obtain the percent of getresponse calls that are above a given latency value
getstats (GET POST)query args
percent_above_value = threshold value in millisecsfrom=start time of metrics
eg to find the percent of Stubo responses that take more than 40ms (during the past 30min)
stuboapigetstatspercent_above_value=40ampfrom=-30mins
version 562data
from -30minstarget averageSeries(statstimersstuboaws_cluster1stuboapiget_responselatencymean_90)metric latencyto nowpercent_above_value 40pcent 00
The key value being pcent which in this case is 00
2124 Jump to Bookmark 65
Stub-O-Matic Documentation Release 0X beta
66 Chapter 21 Stub-O-Matic API v1
CHAPTER 22
Examples
Lots of example usage of Stubo can be found under staticcmdstests The folders under here contain various cmdscripts and their associated files (requests matchers responses) and user exit code examples
You can run these scripts from the Manage page
221 Templates Date Rolling
Shows how you can roll a date using the difference of recorded and play dates
Run staticcmdsteststemplatesdaterollfirstcommands
222 Stateful stubs
Shows how the same request can return different responses
Run staticcmdstestsstateconversecommands
223 Auto Mangling with user exits
Shows response mangling
Run staticcmdstestsextauto_mangleresponseresponseall
Shows skipping XML elements from the matching process
Run staticcmdstestsextauto_mangleskip_xml_elements1all
Shows skipping XML attributes from the matching process
Run staticcmdstestsextauto_mangleskip_xml_attrs1all
Shows an extractor modifing the value of an XPATH result to only match on part of an element value
Run staticcmdstestsextauto_mangleembeddedembeddedall
Shows combined matcher amp response mangling
67
Stub-O-Matic Documentation Release 0X beta
Run staticcmdstestsextauto_mangleall1all
224 user exit using XSLT
Example showing how the same request but with different environmental data can be transformed by lsquotemplatingrsquothe env values in the matcher amp response during putstub and transforming it with XSLT using the env values fromrequest_text during getresponse
Run staticcmdstestsextxlstfirstcommands
225 User exit using a cache
Run staticcmdstestsextcache1commands
68 Chapter 22 Examples
CHAPTER 23
Clients (Stubo Integrator)
A Stubo integrator is responsible for diverting requests destined for real back-end services to Stub-O-Matic As sucha Stubo integrator can take many forms depending on the needs language and architecture of the System Under Test(SUT)
Regardless of the SUT architecture calls to Stubo are REST-ful (ish) HTTP
231 Test data Setup and Teardown
The basic flow for tests which require test data is
1 setup data
2 run Tests
3 teardown data
69
Stub-O-Matic Documentation Release 0X beta
This process can easily be automated using Stubo and your favourite test tool Any test tool capable of making HTTPcalls can use Stubo commands directly or via Stubo command files to setup and teardown test data
Loading stubs is typically accomplished by triggering a command file
httpstubo-serverstubodefaultexecCmdscmdFile=httpURL_OF_FILEcommands
A clever touch is to add a beginsessionmode=playback command to the end of the command file making the stubsready to be used
Data setup and teardown need not involve the Stubo Integrator
232 To Stub or not to Stub
The first decision of an integrator is to send a data request to Stubo or a Live back-end This choice can be made atrun time or when deploying your system Making the choice at run time for each test message is the most powerfuland flexible choice
Use
lsquoxx_stb_useStub=stuborsquo to send data requests to Stubo any other value will use a live back-end
70 Chapter 23 Clients (Stubo Integrator)
Stub-O-Matic Documentation Release 0X beta
233 Playback of Stubs
When an integrator has taken the Stubo path it should create a getresponse call to Stubo The request data must beput in the body of the HTTP POST See the Stubo API documention for detail
The integrator code should accept arguments along with the text of the actual request The one mandatory argumentis the Stubo lsquosessionrsquo A good design is to pass through to Stubo any arguments with an agreed prefix (making sure tostrip off the prefix before passing them through)
For example
xx_stb_session becomes session
xx_stb_ext_module becomes ext_module
Other arguments are listed in the Stubo API documention for getresponse and putstub
234 Record Stubs
If a getresponse is attempted but the session is in record mode Stubo will respond stating that one should be record-ing The integrator should then create and send a putstub call
233 Playback of Stubs 71
Stub-O-Matic Documentation Release 0X beta
Using getresponse as the default call and falling back to putstub should the session be in record mode the test systemrequests need only contain the Stubo session name The mode of the session in Stubo will drive whether a record orplayback takes place
235 Performance Requirements
The integrator should perform better than the system being tested If your live system should handle 1000 requestsper second at the point of stubbing then the integrator should handle at least 1001 If the live back-ends might take5 seconds to respond ensure the integrator has enough threads available to handle the throughput If you scale thesystem by adding more of something ensure you can add more integrators
236 Error Responses
Any HTTP return code equal to or above 300 indicates a problem There is usually nothing the integrator can do withan error apart from pass it back up the chain and call a human
72 Chapter 23 Clients (Stubo Integrator)
CHAPTER 24
Reference
241 Stub Definition
A stub definition consists of a request response pair The JSON definition is as follows
Note this example show the full set of matching criteria In reality some matchers would not be combined eg xpathand jsonpath
request
method GET|POST|PUT|DELETEbodyPatterns [
contains [ltstatusgtOKltstatusgt]xpath [bookstorebook[pricegt3500]
(findme user httpwwwmycomuserschemainfo httpwwwmycominfoschema)]
jsonpath [bookstorebook]
]urlPath getmeurlPattern getme[0-9]+queryArgs
find mewhen now
headers
Content-Type textxmlX-Custom-Header 1234
response
status 200body ltresponsegtYESltresponsegtheaders Content-Type textxmlX-Custom-Header 1234
Most leaf keys can be negated by adding a lsquorsquo prefix to the key
eg
73
Stub-O-Matic Documentation Release 0X beta
request method GET
74 Chapter 24 Reference
CHAPTER 25
About
Stub-O-Matic supports functional performance and load testing The two (opposing) goals regarding performanceare
1 Scalability the ability to support hundreds or thousands of concurrent stub requests without impacting perfor-mance
2 Configurable delays on stub requests to emulate response times of the systems being stubbed
Efficient use of hardware
bull Test teams can share a Stubo server and by using different URLs for the server essentially have their own virtualStubo instance
75
Stub-O-Matic Documentation Release 0X beta
76 Chapter 25 About
CHAPTER 26
Change History
261 stubo Change History
2611 Changelog
066 (2015-08-10)
bull Fixed bug when due to variable imports from testing packages startup failed
bull Imports refactored modules are now more independent
065 (2015-07-24)
bull Correctly showing weighted delays in manage page 77
bull Moving python package requirements into separate requirements file added production and development re-quirement files to avoid installing unnecessary testing packages in production 78
064 (2015-07-16)
bull Added indexes to database optimized queries to database - manage page is loading quickly now even whenthere are tens of thousands of stubs and tracker collection larger than gt 100 000 records 75
bull Switched to mongo aggregation framework functionality when querying scenarios for their sizes and recorddates 68
bull Added API call to rename scenario (should improve automated testing as it is a solution to several GitHub issuesraised by BA) 71
bull Optimised stub insertion 69
bull Brought back export to commands file new YAML format files are exported in separate directory 64
bull Fixed several bugs that could have been causing memory overflow in Stubo instances 61 62
bull Parameter overrides when uploading stubs
bull ldquoworst latencyrdquo chart
77
Stub-O-Matic Documentation Release 0X beta
063 (2015-06-30)
bull YAML config (12)
bull Fix to support repeating elements within XMLMangler (55)
bull New weighted delay (45)
062
bull Rest support
bull Parameterize session on getexport and execcmds calls (39)
061 (2015-05-11)
bull Return the http status code of execcmds requests (28)
bull Added tracker scenario filter (23)
bull Added stub priority
bull Export runnable scenarios
06 (2015-03-25)
bull Initial cut of open source version
78 Chapter 26 Change History
CHAPTER 27
Indices and tables
bull genindex
bull modindex
bull search
79
- Introduction
- Background
- Applicability
- Stub Basics
- Load and Retrieve Stubs
- Sessions and Scenarios
- Recording Stubs
- Stubo Command File
- Command file (YAML)
- Command Scripting
- Stubo Command File (text format)
- Text Command file
- Command Scripting
- Passing Arguments
- GUI
- Tracker
- Manage
- Analytics
- Integration with Test Tools
- Test steps example
- Stubbing
- Add Delay to a Response
- Matching
- Body contains matching
- Templated Matcher
- Templated Responses
- Request Data in Responses
- Stateful Stubs
- User Exits
- Hooks
- Modules
- Splitting
- Caching Values
- Sharing Stubs
- Communicating with Stub-O-Matic
- GetResponse URLs
- Virtual Stub-O-Matic
- Stub-O-Matic API v1
- execcmds
- getversion
- getstatus
- beginsession
- endsession
- endsessions
- putscenarios
- putstub
- getstublist
- putdelay_policy
- getdelay_policy
- deletedelay_policy
- getresponse
- deletestubs
- getexport
- getstubcount
- putmodule
- getmodulelist
- deletemodule
- deletemodules
- Set Tracking Level
- Blacklist a host URL
- Create Bookmark
- Jump to Bookmark
- Delete Bookmark
- List Bookmarks
- getstats
- Examples
- Templates Date Rolling
- Stateful stubs
- Auto Mangling with user exits
- user exit using XSLT
- User exit using a cache
- Clients (Stubo Integrator)
- Test data Setup and Teardown
- To Stub or not to Stub
- Playback of Stubs
- Record Stubs
- Performance Requirements
- Error Responses
- Reference
- Stub Definition
- About
- Change History
- stubo Change History
- Indices and tables
top related