Winning the fight against test automation

Winning the fight against test automation

Chris Mills Mozilla

Who am I?

‣ Tech writer at Mozilla

‣ Writes about Web APIs on MDN

‣ Heads up the MDN Learning Area

‣ HTML.CSS.JS tinkerer

‣ Accessibility whinge bag

‣ Heavy metal drummer

In this talk

‣ The importance of cross browser testing

‣ What I wrote about it

‣ What I learned

‣ What was easy (and difficult)

Cross browser testing

(Important!)

‣ More than 1BN websites

‣ 3BN web users

‣ 8.1BN connected devices

‣ 24,000 mobile device types

‣ Lots of different browsers

‣ ~20% of users have a disability

Why don’t we?

‣ Less experience of X browser issues

‣ Many courses only teach Chrome

‣ Over-reliance on bleeding edge features

‣ And vendor-prefixed features

Other x-browser issues

‣ Browser vendors can be slow to fix bugs

‣ Some people still use browser sniffing rather than feature detection

And of course

‣ Toolchain/workflow perceived as difficult

So I wrote about it

We had to!

‣ Cross browser testing (MDN)

‣ Beginner-friendly guide to X browser

‣ Also friendly to front-end devs

Fitting it in

‣ What is X browser testing?

‣ Where + when should it happen?

‣ What should you test?

‣ How do you test it?

Solving common problems

‣ HTML

‣ CSS

‣ JavaScript

‣ Accessibility

Automating tests

‣ Task runners

‣ Selenium automation

‣ SauceLabs

‣ SauceLabs + Selenium

What was easy?

The webby stuff

‣ Of course!

‣ HTML/CSS/JavaScript is second nature

‣ We also know our accessibility

‣ And our browsers!

Writing the tests

‣ Selenium test logic is fairly easy (once you’ve got it working)

‣ Writing/running the tests was easy (I used Node); modules seem to have good docs

‣ SauceLabs integration was easy

Accessing WDvar webdriver = require('selenium-webdriver'),

By = webdriver.By,

until = webdriver.until;

var driver = new webdriver.Builder()

.forBrowser('firefox')

.build();

Accessing a pagedriver.get('http://www.google.com');

driver.findElement(By.name('q')).sendKeys('webdriver');

driver.findElement(By.name('btnG')).click();

Running a testdriver.sleep(2000).then(function() {

driver.getTitle().then(function(title) {

if(title === 'webdriver - Google Search') {

console.log('Test passed');

} else {

console.log('Test failed');

}

});

});

Finishing updriver.quit();

Accessing SauceLabsvar webdriver = require('selenium-webdriver'),

By = webdriver.By,

until = webdriver.until,

username = "YOUR-USER-NAME",

accessKey = "YOUR-ACCESS-KEY";

Accessing SauceLabsvar driver = new webdriver.Builder().

withCapabilities({

'browserName': 'chrome',

'platform': 'Windows XP',

'version': '43.0',

'username': username,

'accessKey': accessKey

}).

usingServer("https://" + username + ":" + accessKey +

"@ondemand.saucelabs.com:443/wd/hub").

build();

Updating SauceLabsvar saucelabs = new SauceLabs({

username : "YOUR-USER-NAME",

password : "YOUR-ACCESS-KEY"

});

Accessing SauceLabsdriver.getSession().then(function (sessionid){

driver.sessionID = sessionid.id_;

});

Accessing SauceLabs if(title === 'webdriver - Google Search') {

console.log('Test passed');

var testPassed = true;

} else {

console.log('Test failed');

var testPassed = false;

}

saucelabs.updateJob(driver.sessionID, {

name: 'Google search results page title test',

passed: testPassed

});

What was hard?

Setting up Selenium

‣ Working out what to use in the first place

‣ There’s lots of docs…

‣ …and they can be confusing

Setting up Selenium

‣ Do you use WebDriver, or Selenium Server, or Selenium RC, or Selenium Grid, or… ?

‣ Finding drivers for all the browsers you want to automate

‣ Installing the modules (or whatever) that you need for your server-side environment

‣ Making sure everything is communicating

We could benefit from

‣ Simple guides based on other server-side environments

‣ Better API docs for the SauceLabs API

Finito!

‣ Make the web work for everyone (Hacks)

‣ Cross browser testing (MDN)

‣ slideshare.net/chrisdavidmills

[email protected] @chrisdavidmills