How to test complex SaaS applications - The family july 2014

How to test complex SaaS solutions!!

The Family!16th july 2014

Founder & CTO!!

@guillaumepotier!http://imctobitch.com

By the numbers• 4 years old!• 23 employes and counting!- ~ 10 tech / product!- ~ 10 sales!- ~ 3 support, market, administrative !

• 400+ clients!- Total, Orange, Areva, Air Liquide, Google, SNCF, AXA, Société

Générale, Sanofi, L’Oréal, Crédit Agricole, Danone, Deloitte, Capgemini, Auchan…!

• 4000+ tests!- UT backend, functional backend!- UT javascript, functional javascript!

!!!

!

What we’ll talk about

• Why testing?!• What need to be tested?!• How to test?!• Tools!• Limits..!• Going further!

!

Why testing?

Sometimes shit may happen

Why testing?

Tests are here to prevent that (at least they try..)

Why testing?

Tests also allow you refactor without fearing to break things

Why testing?

Tests are great to rely on other people code (and other people on yours)

Why testing?

Ultimately, tests allow you to be faster!

Why testing?

A little more seriously..!Let’s see a real Parsley.js example here

Why testing?

Why testing?

• 10+ classes!• 2k+ lines!• ~200 tests (UT & functional)!• ran in < 3sec in browser!• ran in < 1sec in CLI!

!!

http://parsleyjs.org/doc/tests.html

Why testing?

UT : validate your methods API and behavior

! it('should have a max validator', function () { expect(parsleyValidator.validate('foo', parsleyValidator.max(10))).to.be(false); expect(parsleyValidator.validate('5', parsleyValidator.max(10))).to.be(true); expect(parsleyValidator.validate('10', parsleyValidator.max(10))).to.be(true); expect(parsleyValidator.validate('42', parsleyValidator.max(10))).to.be(false); });

max: function (value) { return $.extend(new Validator.Assert().LessThanOrEqual(value), { priority: 30 }); }

✓ should have a max validator

Code

Test

Result

Why testing?

Prevent regressions, ensure 3rd party libs consistency

! it('should have a max validator', function () { expect(parsleyValidator.validate('foo', parsleyValidator.max(10))).to.be(false); expect(parsleyValidator.validate('5', parsleyValidator.max(10))).to.be(true); expect(parsleyValidator.validate('10', parsleyValidator.max(10))).to.be(true); expect(parsleyValidator.validate('42', parsleyValidator.max(10))).to.be(false); });

max: function (value) { return $.extend(new Validator.Assert().LessThan(value), { priority: 30 }); }

1) should have a max validator

Code

Test

Result

Why testing?

Why testing?

Why testing?

Fixes bugs found to ensure they’ll never show up again

! it('should have a max validator', function () { expect(parsleyValidator.validate('foo', parsleyValidator.max(10))).to.be(false); expect(parsleyValidator.validate('5', parsleyValidator.max(10))).to.be(true); expect(parsleyValidator.validate('10', parsleyValidator.max(10))).to.be(true); expect(parsleyValidator.validate('42', parsleyValidator.max(10))).to.be(false); expect(parsleyValidator.validate('5', parsleyValidator.max(”10”))).to.be(true); });

✓ should have a max validator

Code

Test

Result

max: function (value) { return $.extend(new Validator.Assert().LessThan(value), { priority: 30, requirementsTransformer: function () { return 'string' === typeof value && !isNaN(value) ? parseInt(value, 10) : value; } }); }

Why testing?

it('should show custom error message with variabilized parameters', function () { $('body').append('<input type="text" id="element" value="bar" data-parsley-minlength="7" data-parsley-minlength-message="foo %s bar"/>'); var parsleyField = $('#element').psly(); parsleyField.validate(); ! expect($('ul#parsley-id-' + parsleyField.__id__ + ' li').text()).to.be('foo 7 bar'); });

Functional test : validate your end-user behavior

Why testing?it('should save some calls for querries already done', function (done) { $('body').append('<input type="text" data-parsley-remote="http://foo.bar" id="element" required name="element" value="foo" />'); var parsleyInstance = $('#element').parsley(); ! sinon.stub($, 'ajax').returns($.Deferred().resolve({}, 'success', { status: 200, state: function () { return 'resolved' } })); parsleyInstance.asyncIsValid() .done(function () { expect($.ajax.calledOnce).to.be(true); expect($.ajax.calledWithMatch({ data: { "element": "foo" } })).to.be(true); $.ajax.restore(); sinon.stub($, 'ajax').returns($.Deferred().reject({ status: 400, state: function () { return 'rejected' } }, 'error', 'error')); ! $('#element').val('bar'); parsleyInstance.asyncIsValid() .fail(function () { expect($.ajax.calledOnce).to.be(true); expect($.ajax.calledWithMatch({ data: { "element": "bar" } })).to.be(true); ! $.ajax.restore(); sinon.stub($, 'ajax').returns($.Deferred().resolve({}, 'success', { status: 200, state: function () { return 'resolved' } })); $('#element').val('foo'); ! parsleyInstance.asyncIsValid() .done(function () { expect($.ajax.callCount).to.be(0); expect($.ajax.calledOnce).to.be(false); $.ajax.restore(); done(); }); }); }); });

What need to be tested?

What need to be tested?

• Application logic (services, algorithms, microapps)!• API responses!• Application behavior, End-user responses!

!

TO  BE  TESTED

What need to be tested?

What need to be tested?

• getters / setters!• already tested 3rd party libraries!• ORM, all exceptions, errors!

!

NOT  TO  BE  TESTED

What need to be tested?

What need to be tested?

100%  coverage!

What need to be tested?

What need to be tested?

What need to be tested?

How to test?

How to test?

How to test?class SynchronizedUserTest extends PHPUnit_Framework_TestCase { /** * @covers ::construct() * @expectedException InvalidArgumentException * @expectedExceptionMessage User and IntercomUser are not the same. */ public function testConstructWithWrongEmails() { $intercomUser = new IntercomUser(1, '[email protected]'); $user = (new User)->setEmail('[email protected]'); $this->setProperty($user, 'id', 1); ! new SynchronizedUser($intercomUser, $user); } ! /** * @covers ::construct() * @expectedException InvalidArgumentException * @expectedExceptionMessage User and IntercomUser are not the same. */ public function testConstructWithWrongIds() { $intercomUser = new IntercomUser(2, '[email protected]'); $user = (new User)->setEmail('[email protected]'); $this->setProperty($user, 'id', 1); ! new SynchronizedUser($intercomUser, $user); } }

How to test?class SynchronizedUser { private $intercomUser; private $user; ! /** * @throws InvalidArgumentException If the user intercom and user wisembly doesn't match */ public function __construct(IntercomUser $intercomUser, User $user) { if ($intercomUser->getUserId() !== $user->getId() || $intercomUser->getEmail() !== mb_strtolower($user->getEmail(), mb_detect_encoding($user->getEmail()))) { throw new InvalidArgumentException('User and IntercomUser are not the same.'); } ! $this->intercomUser = $intercomUser; $this->user = $user; } ! /** * @return IntercomUser */ public function getIntercomUser() { return $this->intercomUser; } ! /** * @return User */ public function getUser() { return $this->user; } }

How to test?

How to test?

Tools

• PHPUnit!• Behat, PHPSpec..!• Mocha, Jasmine, QUnit..!• Karma, PhantomJS, TestEM..!• Travis, Jenkins, Shipper, CodeShip..!

!

Limits

• CSS glitches!• User experience!• Browser compatibility!• Do not take to much time to code / maintain tests!!• Do not have too long test suites!!• Fixtures for functional tests!• Isolate tests among themselves!!

!

Going further

• Script & test deploy scripts..!• SLA & performance tests..!

!