A UTOMATING S ECURITY T ESTS WITH S ELENIUM By Brady Vitrano & Charles Neill Presented to OWASP San Antonio March 20th, 2015.

AUTOMATING SECURITY TESTS WITH SELENIUM

By Brady Vitrano & Charles Neill

Presented to OWASP San Antonio

March 20th, 2015

Who Are We?

2

Charles NeillSecurity Developer

Brady VitranoLead of Quality Engineering

• The goals• Selenium/Tools/Language Introduction • Security Engineering Introduction • Create and run security tests• Scalable Testing with the Grid• Takeaways • Q&A• Git Repo - https://github.com/cneill/selenium-security-stuff

Agenda

• Understand Selenium framework for UI automation testing• Learn why Selenium is a useful framework for frontend security testing• Learn to create simple function test cases using Selenium• Learn to create simple security testing cases using Selenium

The Goals

What is Selenium?

• What is Selenium?–Earth Metal

–Atomic Number: 34

–Atomic Weight: 78.96

–Tool to control web browsers and devices

• Selenium Modes–WebDriver API

• Support Remote Browsers

–Selenium IDE Recorder• Runs locally

• Web Driver Pros–Scripts written to perform browser actions to

simulate web user

–Tests against various browsers and devices

–Flexible to handle frequent code changes

–Watch scripts run against live browser

–Scalable with Selenium Grid

• Web Driver Cons–Simulates user actions but does not support

scrolling

–Must hack shortcomings with Javascript

–WebDriver tends to be out of date with frequent browser updates

Browser Automation with Selenium

• IDE Pros–Quick and temporary solution

• IDE Cons–Manual Process (SLOW)

–Requires tons of maintenance

–Breaks frequently do to outdated tests

–Does not run remotely

• IDE NOT RECOMMENDED

• Selenium Webdriver–REST API based works with various browsers and devices

–JSON Wire Protocol• https://code.google.com/p/selenium/wiki/JsonWireProtocol

–W3 WebDriver (Draft 11 Feb 2015)• https://w3c.github.io/webdriver/webdriver-spec.html

• Programming Languages–Python – (Covered Today)

–Ruby (Merlot – Rackspace Gem)

–Javascript (Protractor)

–Java (???)

Technical Overview

8

Very basics of a webpage functionality test include:

• pip install selenium

• Visiting the webpage of interest

• Accessing the elements on the page

Using Selenium / Examples

www.rackspace.com

Explanation:

Here we simply create a Firefox browser object. This will cause a Firefox window to launch.

Next we tell Firefox to navigate to http://seleniumhq.org/ Which will load the requested website.

9

Selenium / Examples

www.rackspace.com

Example 1:

• open a new Firefox browser

• load the Yahoo homepage

• search for “seleniumhq”

• close the browser

10

Selenium / Examples - Page Elements

www.rackspace.com

Elements can be accessed in many ways:

• Element type: input, button

• Element attribute: name, id, value

• Xpath

User actions that can be emulated:

• Click

• Filling out text fields (sending keys)

11

Selenium / Examples – Inspecting Elements

www.rackspace.com

12

• There are several different ways the webdriver can find HTML elements:

Selenium / Examples – How to access page elements

www.rackspace.com

One of the most commonly used assessors is browser.find_element_by_css_selector()

This works off of CSS selectors (similar to sizzle/jQuery's selection system)

IPython is a helpful way to explore the Selenium API. Tab completion will help you find interesting methods

• Part of Quality Engineering• Focus on security testing of different practice areas • Integrate with project team as security resource• Provide security testing

– Infrastructure security testing

–Web application security testing

–API security testing

Security Engineering Introduction

• Injection (such as SQL, OS, LDAP injection)• Broken Authentication and Session Management • Cross Site Scripting (XSS)• Insecure Direct Object References• Security Misconfiguration • Sensitive Data Exposure• Missing Function Level Access Control• Cross-Site Request Forgery (CSRF)• Using Components with Known Vulnerabilities• Unvalidated Redirects and Forwards

Common Security Vulnerabilities for Web Applications

SQL Injection

select * from Users where username = 'submittedUser' and password = 'submittedPassword';

username=adminpassword=bad' or 1=1--

select * from Users where username = 'admin' and password = 'bad ' or 1=1--;

•Authentication bypass •Read sensitive data from the database

•Modify database data•Execute administrative operations

•Local File system access•Run operation system command•…

XSS

<?php$name = $_GET['name'];echo "Welcome $name<br>";echo "<a href="http://xssattackexamples.com/">Click to Download</a>";?>

index.php?name=guest<script>alert('attacked')</script>

Welcome guest <script>alert(‘attacked’)</script>

•Session stealing•Malware installation•Phishing•HTML5 Storage Compromise•Compromising Credentials•Cross Site Request Forgery Attack

•Cookie Stealing• Identity Theft

•…

17

Another tool to learn…

www.rackspace.com

18

• Don't we want to just throw some fuzz strings at the app and hope it returns a 500 error?– We want to eliminate false positives, because we might not be around to

watch the test execution and dig through it manually (e.g. if this is being run in a Jenkins gating job)

• Writing test cases is harder than just loading a wordlist into BURP– Sure, the first time! But once you write some code, you can re-use it over

and over again, and you can define success/failure in a more detailed way for each test

Why Scripted Tests?

www.rackspace.com

19

• Run tests with real-world browsers, and inspect tests in the browser if you want

• Flexible test running – you can run tests locally, or spin up a whole cluster of headless nodes to test in parallel

• Ability to manipulate the page in a more "natural" way with Selenium, unlike tools that don't emulate or control a browser• Manipulate the DOM• Work with JavaScript directly

Why Selenium?

www.rackspace.com

20

• Bindings in many languages (Python, Ruby, Java…)

• Catch pop-ups as they occur – this is a great way to verify XSS

• Search the DOM with CSS selectors (similar to jQuery)

• Inject JavaScript, cookies, and browser extensions on-the-fly

Why Selenium? (Cont'd.)

www.rackspace.com

21

• If you're running thousands of tests, Selenium will take significantly longer than something like cURL

•If you're just doing HTTP requests and searching for regexes in the page source code, this can be done much faster by other means– For this reason, Selenium isn't particularly well-suited to API testing

Why NOT Selenium?

www.rackspace.com

22

Security Automation Demo!

www.rackspace.com

• Preinstalled on Vagrant

• SSH into Vagrant Box– vagrant ssh

• Start Grid– sudo dsgrid start

• Add Multiple Firefox Nodes– sudo dsgrid nodes add firefox

– sudo dsgrid nodes add firefox

• Verify Selenium Hub is Running– http://localhost:49044/grid/console

Selenium Grid

Instead of this:

Run Automation Against the Grid

Use this:

• https://w3c.github.io/webdriver/webdriver-spec.html• http://www.seleniumhq.org/• https://github.com/cneill/selenium-security-stuff• OWASP Top Ten Project: https://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project

• SQL injection: https://www.owasp.org/index.php/SQL_Injection• Cross Site Scripting (XSS): https://www.owasp.org/index.php/Cross-site_Scripting_%28XSS%29

Reference

Questions?

[email protected]@RACKSPACE.COM

THANK YOU

RACKSPACE® | 1 FANATICAL PLACE, CITY OF WINDCREST | SAN ANTONIO, TX 78218

US SALES: 1-800-961-2888 | US SUPPORT: 1-800-961-4454 | WWW.RACKSPACE.COM

© RACKSPACE LTD. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED STATES AND OTHER COUNTRIES. | WWW.RACKSPACE.COM